You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "adriangb (via GitHub)" <gi...@apache.org> on 2026/02/15 04:46:25 UTC
[PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
adriangb opened a new pull request, #20363:
URL: https://github.com/apache/datafusion/pull/20363
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904490794
Wonder if I'm infinite looping it or something :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904954170
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903920732
Hm it seems stuck again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20363: (Test) Advanced adaptive filter selectivity evaluation
URL: https://github.com/apache/datafusion/pull/20363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3955645564
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960343455
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229996963
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4229961271)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³βββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.20 / 4.47 Β±6.39 / 17.25 ms β 1.18 / 4.43 Β±6.34 / 17.10 ms β no change β
β QQuery 1 β 15.70 / 15.95 Β±0.18 / 16.18 ms β 15.69 / 15.97 Β±0.19 / 16.19 ms β no change β
β QQuery 2 β 43.78 / 44.28 Β±0.44 / 45.06 ms β 44.12 / 44.80 Β±0.40 / 45.40 ms β no change β
β QQuery 3 β 44.02 / 46.74 Β±1.63 / 48.18 ms β 44.63 / 46.27 Β±1.14 / 48.07 ms β no change β
β QQuery 4 β 288.34 / 300.47 Β±11.10 / 318.83 ms β 284.73 / 293.91 Β±6.25 / 301.96 ms β no change β
β QQuery 5 β 345.38 / 350.65 Β±3.13 / 353.80 ms β 340.37 / 349.23 Β±5.35 / 354.52 ms β no change β
β QQuery 6 β 5.92 / 6.14 Β±0.14 / 6.32 ms β 6.34 / 8.44 Β±2.61 / 13.48 ms β 1.37x slower β
β QQuery 7 β 21.81 / 22.83 Β±1.46 / 25.71 ms β 22.25 / 22.47 Β±0.21 / 22.78 ms β no change β
β QQuery 8 β 415.69 / 426.71 Β±7.42 / 434.64 ms β 424.78 / 431.49 Β±5.36 / 437.86 ms β no change β
β QQuery 9 β 648.47 / 657.12 Β±5.93 / 663.73 ms β 637.74 / 652.94 Β±8.62 / 660.95 ms β no change β
β QQuery 10 β 116.89 / 120.09 Β±4.12 / 128.04 ms β 119.95 / 123.64 Β±3.95 / 130.69 ms β no change β
β QQuery 11 β 131.39 / 134.19 Β±2.94 / 139.51 ms β 131.85 / 134.42 Β±1.74 / 136.84 ms β no change β
β QQuery 12 β 371.92 / 381.11 Β±6.78 / 389.37 ms β 371.25 / 385.03 Β±11.68 / 398.31 ms β no change β
β QQuery 13 β 482.14 / 508.22 Β±17.13 / 533.52 ms β 495.10 / 513.96 Β±15.27 / 535.77 ms β no change β
β QQuery 14 β 376.74 / 384.73 Β±5.56 / 393.34 ms β 385.63 / 391.47 Β±3.76 / 396.65 ms β no change β
β QQuery 15 β 348.50 / 366.79 Β±9.93 / 374.62 ms β 364.39 / 396.28 Β±57.92 / 511.92 ms β 1.08x slower β
β QQuery 16 β 714.56 / 724.66 Β±12.34 / 747.72 ms β 720.54 / 734.78 Β±17.77 / 769.78 ms β no change β
β QQuery 17 β 705.03 / 716.64 Β±8.91 / 726.57 ms β 706.73 / 726.55 Β±13.68 / 740.18 ms β no change β
β QQuery 18 β 1411.67 / 1478.04 Β±40.28 / 1537.16 ms β 1435.24 / 1469.34 Β±27.73 / 1511.89 ms β no change β
β QQuery 19 β 37.30 / 41.06 Β±3.13 / 46.57 ms β 36.16 / 38.51 Β±2.41 / 42.95 ms β +1.07x faster β
β QQuery 20 β 722.79 / 744.98 Β±23.81 / 778.40 ms β 714.71 / 726.49 Β±15.04 / 755.41 ms β no change β
β QQuery 21 β 728.66 / 748.77 Β±10.95 / 759.64 ms β 787.84 / 810.58 Β±14.92 / 832.04 ms β 1.08x slower β
β QQuery 22 β 1142.36 / 1166.32 Β±12.96 / 1180.36 ms β 1139.09 / 1156.37 Β±14.92 / 1174.65 ms β no change β
β QQuery 23 β 266.19 / 268.77 Β±2.38 / 272.00 ms β 337.12 / 344.80 Β±7.08 / 357.50 ms β 1.28x slower β
β QQuery 24 β 93.64 / 94.69 Β±0.92 / 95.78 ms β 104.25 / 105.20 Β±0.88 / 106.77 ms β 1.11x slower β
β QQuery 25 β 177.03 / 180.69 Β±3.61 / 186.69 ms β 174.88 / 178.68 Β±3.25 / 183.53 ms β no change β
β QQuery 26 β 110.32 / 113.92 Β±2.96 / 118.76 ms β 112.08 / 116.62 Β±3.14 / 120.79 ms β no change β
β QQuery 27 β 975.50 / 992.16 Β±15.09 / 1015.27 ms β 970.00 / 981.63 Β±15.95 / 1013.19 ms β no change β
β QQuery 28 β 3417.46 / 3439.03 Β±12.14 / 3450.64 ms β 3429.59 / 3441.92 Β±9.93 / 3451.99 ms β no change β
β QQuery 29 β 49.51 / 55.33 Β±5.36 / 64.17 ms β 51.06 / 55.32 Β±4.42 / 63.37 ms β no change β
β QQuery 30 β 360.73 / 365.29 Β±5.01 / 373.55 ms β 369.74 / 377.17 Β±4.40 / 382.23 ms β no change β
β QQuery 31 β 349.96 / 368.81 Β±10.94 / 380.24 ms β 371.53 / 381.70 Β±8.24 / 395.10 ms β no change β
β QQuery 32 β 1133.74 / 1270.19 Β±107.89 / 1405.52 ms β 1059.86 / 1127.42 Β±79.88 / 1273.66 ms β +1.13x faster β
β QQuery 33 β 1453.11 / 1477.72 Β±24.41 / 1519.30 ms β 1481.86 / 1494.20 Β±9.90 / 1508.54 ms β no change β
β QQuery 34 β 1472.11 / 1490.56 Β±15.57 / 1518.74 ms β 1491.54 / 1505.17 Β±8.56 / 1518.09 ms β no change β
β QQuery 35 β 390.91 / 400.12 Β±11.55 / 422.31 ms β 395.86 / 411.29 Β±8.49 / 418.07 ms β no change β
β QQuery 36 β 98.75 / 104.37 Β±4.81 / 110.76 ms β 101.17 / 107.97 Β±3.50 / 111.04 ms β no change β
β QQuery 37 β 53.40 / 56.55 Β±2.98 / 62.00 ms β 53.38 / 57.64 Β±2.64 / 61.59 ms β no change β
β QQuery 38 β 54.63 / 59.09 Β±2.78 / 62.63 ms β 55.57 / 59.14 Β±3.14 / 64.24 ms β no change β
β QQuery 39 β 170.07 / 177.93 Β±7.99 / 187.86 ms β 176.85 / 187.11 Β±8.58 / 197.51 ms β 1.05x slower β
β QQuery 40 β 29.49 / 30.57 Β±0.64 / 31.47 ms β 29.20 / 32.49 Β±2.91 / 37.42 ms β 1.06x slower β
β QQuery 41 β 25.98 / 27.30 Β±1.36 / 29.47 ms β 24.57 / 27.30 Β±1.62 / 29.20 ms β no change β
β QQuery 42 β 22.43 / 22.71 Β±0.24 / 23.14 ms β 22.66 / 22.99 Β±0.29 / 23.40 ms β no change β
βββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 20386.75ms β
β Total Time (filter-pushdown-dynamic-bytes) β 20493.12ms β
β Average Time (HEAD) β 474.11ms β
β Average Time (filter-pushdown-dynamic-bytes) β 476.58ms β
β Queries Faster β 2 β
β Queries Slower β 7 β
β Queries with No Change β 34 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 103.1s |
| Peak memory | 41.4 GiB |
| Avg memory | 29.3 GiB |
| CPU user | 926.5s |
| CPU sys | 98.0s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 103.7s |
| Peak memory | 44.0 GiB |
| Avg memory | 30.0 GiB |
| CPU user | 939.1s |
| CPU sys | 90.5s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3976650497
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.48 ms β 2.50 ms β no change β
β QQuery 1 β 202.87 ms β 202.21 ms β no change β
β QQuery 2 β 556.07 ms β 555.89 ms β no change β
β QQuery 3 β 595.40 ms β 616.72 ms β no change β
β QQuery 4 β 3272.79 ms β 3404.51 ms β no change β
β QQuery 5 β 4213.42 ms β 4194.45 ms β no change β
β QQuery 6 β 130.31 ms β 143.93 ms β 1.10x slower β
β QQuery 7 β 245.12 ms β 245.87 ms β no change β
β QQuery 8 β 4768.10 ms β 4736.28 ms β no change β
β QQuery 9 β 7067.50 ms β 7246.34 ms β no change β
β QQuery 10 β 1668.88 ms β 1668.57 ms β no change β
β QQuery 11 β 1902.86 ms β 1882.05 ms β no change β
β QQuery 12 β 4639.93 ms β 4585.60 ms β no change β
β QQuery 13 β 6472.35 ms β 6462.94 ms β no change β
β QQuery 14 β 4519.22 ms β 4625.05 ms β no change β
β QQuery 15 β 3745.53 ms β 3721.30 ms β no change β
β QQuery 16 β 7542.37 ms β 7562.28 ms β no change β
β QQuery 17 β 7436.60 ms β 7463.26 ms β no change β
β QQuery 18 β 13100.52 ms β 13418.25 ms β no change β
β QQuery 19 β 468.03 ms β 468.91 ms β no change β
β QQuery 20 β 7743.68 ms β 7920.65 ms β no change β
β QQuery 21 β 10488.79 ms β 10828.40 ms β no change β
β QQuery 22 β 23651.68 ms β 14221.22 ms β +1.66x faster β
β QQuery 23 β 1496.39 ms β 3570.17 ms β 2.39x slower β
β QQuery 24 β 396.25 ms β 468.96 ms β 1.18x slower β
β QQuery 25 β 2545.09 ms β 2572.58 ms β no change β
β QQuery 26 β 658.31 ms β 544.82 ms β +1.21x faster β
β QQuery 27 β 12840.04 ms β 12896.40 ms β no change β
β QQuery 28 β 97608.45 ms β 98514.00 ms β no change β
β QQuery 29 β 3949.86 ms β 3938.53 ms β no change β
β QQuery 30 β 4682.33 ms β 4593.68 ms β no change β
β QQuery 31 β 5932.81 ms β 5488.30 ms β +1.08x faster β
β QQuery 32 β 29124.65 ms β 26288.61 ms β +1.11x faster β
β QQuery 33 β 16714.15 ms β 16561.61 ms β no change β
β QQuery 34 β 17756.56 ms β 16563.23 ms β +1.07x faster β
β QQuery 35 β 6628.58 ms β 6380.75 ms β no change β
β QQuery 36 β 186.51 ms β 182.13 ms β no change β
β QQuery 37 β 102.15 ms β 130.73 ms β 1.28x slower β
β QQuery 38 β 105.32 ms β 113.17 ms β 1.07x slower β
β QQuery 39 β 287.33 ms β 273.34 ms β no change β
β QQuery 40 β 69.06 ms β 63.20 ms β +1.09x faster β
β QQuery 41 β 62.73 ms β 46.02 ms β +1.36x faster β
β QQuery 42 β 44.76 ms β 59.52 ms β 1.33x slower β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββ
β Benchmark Summary β β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 315625.81ms β
β Total Time (filter-pushdown-dynamic-bytes) β 305426.93ms β
β Average Time (HEAD) β 7340.14ms β
β Average Time (filter-pushdown-dynamic-bytes) β 7102.95ms β
β Queries Faster β 7 β
β Queries Slower β 6 β
β Queries with No Change β 30 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315
run benchmark tpch
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979873191
It does seem to have gotten stuck again. Iβm working on a system that can run benchmarks in parallel and wonβt get borked like this. I think itβs almost ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979877703
@Dandandan i deleted your comment to stop the spam. I honestly donβt know whatβs wrong with this PR or the runner. I think I should close it and open a new one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3908488215
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3908487871).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3905800281.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3905800281` |
| `20363_3905800999.sh` | adriangb | default (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3905800999` |
| `20378_3906877330.sh` | Dandandan | clickbench_partitioned | `https://github.com/apache/datafusion/pull/20378#issuecomment-3906877330` |
| `20378_3906878700.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20378#issuecomment-3906878700` |
| `arrow-9414-3907223390.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9414#issuecomment-3907223390` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904489674
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977549086
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (8f9827d548c0a2a924bbe7073aabf6b705213d49) to acec058cb5e32c7a339280d7b58ebe58d8b38e8b [diff](https://github.com/apache/datafusion/compare/acec058cb5e32c7a339280d7b58ebe58d8b38e8b..8f9827d548c0a2a924bbe7073aabf6b705213d49) using: tpcds
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979872821
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979870614
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977022885
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.53 ms β 2.56 ms β no change β
β QQuery 1 β 52.52 ms β 54.56 ms β no change β
β QQuery 2 β 134.09 ms β 131.42 ms β no change β
β QQuery 3 β 155.11 ms β 155.37 ms β no change β
β QQuery 4 β 1019.55 ms β 1020.87 ms β no change β
β QQuery 5 β 1239.03 ms β 1236.72 ms β no change β
β QQuery 6 β 18.39 ms β 12.87 ms β +1.43x faster β
β QQuery 7 β 64.94 ms β 66.71 ms β no change β
β QQuery 8 β 1359.74 ms β 1408.72 ms β no change β
β QQuery 9 β 1667.75 ms β 1698.57 ms β no change β
β QQuery 10 β 477.85 ms β 467.52 ms β no change β
β QQuery 11 β 520.16 ms β 517.58 ms β no change β
β QQuery 12 β 1363.33 ms β 1376.72 ms β no change β
β QQuery 13 β 2030.23 ms β 2023.42 ms β no change β
β QQuery 14 β 1379.70 ms β 1392.96 ms β no change β
β QQuery 15 β 1123.89 ms β 1185.98 ms β 1.06x slower β
β QQuery 16 β 2421.92 ms β 2483.67 ms β no change β
β QQuery 17 β 2422.95 ms β 2456.39 ms β no change β
β QQuery 18 β 4751.71 ms β 4655.63 ms β no change β
β QQuery 19 β 138.35 ms β 137.89 ms β no change β
β QQuery 20 β 1768.46 ms β 1699.27 ms β no change β
β QQuery 21 β 2160.10 ms β 2177.55 ms β no change β
β QQuery 22 β 3781.01 ms β 3010.13 ms β +1.26x faster β
β QQuery 23 β 1061.50 ms β 1208.62 ms β 1.14x slower β
β QQuery 24 β 234.82 ms β 240.56 ms β no change β
β QQuery 25 β 619.43 ms β 632.54 ms β no change β
β QQuery 26 β 342.79 ms β 314.29 ms β +1.09x faster β
β QQuery 27 β 2761.25 ms β 2392.49 ms β +1.15x faster β
β QQuery 28 β 24123.17 ms β 21911.46 ms β +1.10x faster β
β QQuery 29 β 977.85 ms β 972.66 ms β no change β
β QQuery 30 β 1230.57 ms β 1262.23 ms β no change β
β QQuery 31 β 1303.00 ms β 1279.77 ms β no change β
β QQuery 32 β 4169.85 ms β 4053.36 ms β no change β
β QQuery 33 β 4966.37 ms β 5063.13 ms β no change β
β QQuery 34 β 5695.37 ms β 5708.51 ms β no change β
β QQuery 35 β 1810.30 ms β 1840.47 ms β no change β
β QQuery 36 β 177.06 ms β 171.71 ms β no change β
β QQuery 37 β 85.83 ms β 83.13 ms β no change β
β QQuery 38 β 87.55 ms β 82.76 ms β +1.06x faster β
β QQuery 39 β 278.31 ms β 273.14 ms β no change β
β QQuery 40 β 57.01 ms β 44.46 ms β +1.28x faster β
β QQuery 41 β 50.14 ms β 32.69 ms β +1.53x faster β
β QQuery 42 β 36.92 ms β 37.29 ms β no change β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 80122.42ms β
β Total Time (filter-pushdown-dynamic-bytes) β 76978.37ms β
β Average Time (HEAD) β 1863.31ms β
β Average Time (filter-pushdown-dynamic-bytes) β 1790.19ms β
β Queries Faster β 8 β
β Queries Slower β 2 β
β Queries with No Change β 33 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947414622
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (aaa75f256caa1f3c66d9c91ae563f2b2635c563d) to b9328b9734534c8e2ce92b00c5368f88956bf5c3 [diff](https://github.com/apache/datafusion/compare/b9328b9734534c8e2ce92b00c5368f88956bf5c3..aaa75f256caa1f3c66d9c91ae563f2b2635c563d) using: tpcds
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905144070
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (b2c8d8fb7ad987c11e554725d9310e98cb09ae6f) to 53b0ffb93d5fb662506439b6f1ffc34f2d7684ec [diff](https://github.com/apache/datafusion/compare/53b0ffb93d5fb662506439b6f1ffc34f2d7684ec..b2c8d8fb7ad987c11e554725d9310e98cb09ae6f) using: tpch
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230010838
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230007606)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4230007606-1094-4nrx2 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (290784a7a7630ae9bb0ba4f596110211024038a3) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..290784a7a7630ae9bb0ba4f596110211024038a3) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977534423
Oh no I think it hit an infinite loop π€£
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230263398
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230259802)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4230259802-1097-gvvx6 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (72f296f766dbbc2a81c64b8c19f49ec73a39f96d) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..72f296f766dbbc2a81c64b8c19f49ec73a39f96d) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230259644
run benchmark clickbench_partitioned
```yaml
env:
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS: true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS: true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979871237
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979873409
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979988580
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (8f9827d548c0a2a924bbe7073aabf6b705213d49) to acec058cb5e32c7a339280d7b58ebe58d8b38e8b [diff](https://github.com/apache/datafusion/compare/acec058cb5e32c7a339280d7b58ebe58d8b38e8b..8f9827d548c0a2a924bbe7073aabf6b705213d49) using: tpcds
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905143729
run benchmark tpch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903918629
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905800566
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (bcd36f06ad109953b48686a1d0bdb3a22f1518bd) to 53b0ffb93d5fb662506439b6f1ffc34f2d7684ec [diff](https://github.com/apache/datafusion/compare/53b0ffb93d5fb662506439b6f1ffc34f2d7684ec..bcd36f06ad109953b48686a1d0bdb3a22f1518bd) using: tpcds
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3929034564
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3929033989).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20417_3919093898.sh` | Dandandan | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919093898` |
| `20417_3919839904.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919839904` |
| `arrow-9118-3925061519.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9118#issuecomment-3925061519` |
| `20363_3927114392.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392` |
| `arrow-9441-3928995639.sh` | Dandandan | boolean_kernels | `https://github.com/apache/arrow-rs/pull/9441#issuecomment-3928995639` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947388259
π€ Hi @adriangb, thanks for the request (https://github.com/apache/datafusion/pull/20363#issuecomment-3947387517).
[`scrape_comments.py`](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/scrape_comments.py) only supports whitelisted benchmarks.
- Standard: clickbench_1, clickbench_extended, clickbench_partitioned, clickbench_pushdown, external_aggr, tpcds, tpch, tpch10, tpch_mem, tpch_mem10
- Criterion: aggregate_query_sql, aggregate_vectorized, case_when, character_length, in_list, left, plan_reuse, range_and_generate_series, replace, reset_plan_states, sort, sql_planner, strpos, substr_index, with_hashes
Please choose one or more of these with `run benchmark <name>` or `run benchmark <name1> <name2>...`
You can also set environment variables on subsequent lines:
```
run benchmark tpch_mem
DATAFUSION_RUNTIME_MEMORY_LIMIT=1G
```
Unsupported benchmarks: tcpds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947387517
run benchmark tcpds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904526739
> > Wonder if I'm infinite looping it or something :(
>
> Yes I think previously it got stuck during infinite loops / extremely long running tasks.
My bad Iβll try to add a PR to have timeouts and a cancel command
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905144081
I restarted the job runner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927154316
benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927149208
benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947413716
run benchmark tpcds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3950589190
> Yeah, seems like itβs always tpcds? I donβt think itβs this branch necessarily, it got stuck on your branch earlier and this one has been pretty much completely rewritten since last time it got stuck here.
Hmmm could be...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945514401
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947324308
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947177559
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3947177189).
No pending jobs in `jobs/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977534495
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3976567342
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (3c076829b5ecc0c8c26a83dec3c1f7a802190549) to acec058cb5e32c7a339280d7b58ebe58d8b38e8b [diff](https://github.com/apache/datafusion/compare/acec058cb5e32c7a339280d7b58ebe58d8b38e8b..3c076829b5ecc0c8c26a83dec3c1f7a802190549) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3976985628
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (d61fb4997d2ec5f0e4fb3f97e66beab782cbbd32) to acec058cb5e32c7a339280d7b58ebe58d8b38e8b [diff](https://github.com/apache/datafusion/compare/acec058cb5e32c7a339280d7b58ebe58d8b38e8b..d61fb4997d2ec5f0e4fb3f97e66beab782cbbd32) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979870929
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979874069
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4756039237
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230007775
run benchmark clickbench_partitioned
```yaml
env:
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS: true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS: true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905800999
run benchmarks
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904523263
> Wonder if I'm infinite looping it or something :(
Yes I think previously it got stuck during infinite loops / extremely long running tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904954291
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3904954170).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3903261774.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903261774` |
| `20363_3903262814.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903262814` |
| `20365_3903537986.sh` | Dandandan | tpch tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903537986` |
| `20365_3903568877.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903568877` |
| `arrow-9414-3904516323.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9414#issuecomment-3904516323` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3908487871
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945794062
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3945793206).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20481_3943833990.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20481#issuecomment-3943833990` |
| `19728_3944810475.sh` | alamb | default | `https://github.com/apache/datafusion/pull/19728#issuecomment-3944810475` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3956271177
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945200889
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (26111e94513e9c5c92d2058f5e33033361669c14) to 9660c9874315354ff22245699785f5f77841be80 [diff](https://github.com/apache/datafusion/compare/9660c9874315354ff22245699785f5f77841be80..26111e94513e9c5c92d2058f5e33033361669c14) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927157700
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927566594
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3927566224).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20417_3919093898.sh` | Dandandan | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919093898` |
| `20417_3919839904.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919839904` |
| `arrow-9118-3925061519.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9118#issuecomment-3925061519` |
| `20363_3927114392.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977532238
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977533305
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977533802
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977534241
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979873146
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979875342
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3955225206
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979872541
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979876368
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977479441
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.61 ms β 2.62 ms β no change β
β QQuery 1 β 52.67 ms β 52.19 ms β no change β
β QQuery 2 β 130.30 ms β 137.37 ms β 1.05x slower β
β QQuery 3 β 155.69 ms β 154.25 ms β no change β
β QQuery 4 β 995.99 ms β 1049.33 ms β 1.05x slower β
β QQuery 5 β 1225.07 ms β 1252.44 ms β no change β
β QQuery 6 β 18.88 ms β 6.42 ms β +2.94x faster β
β QQuery 7 β 66.71 ms β 59.76 ms β +1.12x faster β
β QQuery 8 β 1359.18 ms β 1426.13 ms β no change β
β QQuery 9 β 1728.10 ms β 1773.00 ms β no change β
β QQuery 10 β 478.15 ms β 466.47 ms β no change β
β QQuery 11 β 519.96 ms β 509.89 ms β no change β
β QQuery 12 β 1378.92 ms β 1196.03 ms β +1.15x faster β
β QQuery 13 β 2085.93 ms β 2020.55 ms β no change β
β QQuery 14 β 1407.29 ms β 1416.22 ms β no change β
β QQuery 15 β 1158.02 ms β 1208.20 ms β no change β
β QQuery 16 β 2458.30 ms β 2505.55 ms β no change β
β QQuery 17 β 2437.31 ms β 2508.15 ms β no change β
β QQuery 18 β 5200.12 ms β 4803.78 ms β +1.08x faster β
β QQuery 19 β 143.20 ms β 148.09 ms β no change β
β QQuery 20 β 1817.30 ms β 1890.48 ms β no change β
β QQuery 21 β 2292.85 ms β 2254.91 ms β no change β
β QQuery 22 β 3905.74 ms β 3189.17 ms β +1.22x faster β
β QQuery 23 β 1036.76 ms β 1780.00 ms β 1.72x slower β
β QQuery 24 β 243.93 ms β 203.19 ms β +1.20x faster β
β QQuery 25 β 617.69 ms β 454.12 ms β +1.36x faster β
β QQuery 26 β 333.02 ms β 251.67 ms β +1.32x faster β
β QQuery 27 β 2865.84 ms β 2495.76 ms β +1.15x faster β
β QQuery 28 β 24457.79 ms β 23208.99 ms β +1.05x faster β
β QQuery 29 β 973.84 ms β 979.94 ms β no change β
β QQuery 30 β 1265.61 ms β 1274.79 ms β no change β
β QQuery 31 β 1312.85 ms β 1332.77 ms β no change β
β QQuery 32 β 4246.97 ms β 4028.18 ms β +1.05x faster β
β QQuery 33 β 5258.91 ms β 5134.74 ms β no change β
β QQuery 34 β 5599.48 ms β 5680.80 ms β no change β
β QQuery 35 β 1886.69 ms β 1865.47 ms β no change β
β QQuery 36 β 174.06 ms β 158.88 ms β +1.10x faster β
β QQuery 37 β 86.23 ms β 78.77 ms β +1.09x faster β
β QQuery 38 β 89.00 ms β 92.51 ms β no change β
β QQuery 39 β 279.59 ms β 306.23 ms β 1.10x slower β
β QQuery 40 β 58.51 ms β 47.46 ms β +1.23x faster β
β QQuery 41 β 49.26 ms β 33.71 ms β +1.46x faster β
β QQuery 42 β 36.88 ms β 38.29 ms β no change β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 81891.19ms β
β Total Time (filter-pushdown-dynamic-bytes) β 79477.25ms β
β Average Time (HEAD) β 1904.45ms β
β Average Time (filter-pushdown-dynamic-bytes) β 1848.31ms β
β Queries Faster β 15 β
β Queries Slower β 4 β
β Queries with No Change β 24 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977533562
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977534019
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979870353
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230007606
run benchmark clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229963173
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4229959251)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4229959251-1092-g5tm7 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (f8ee95523252ee7b59d52c411735c6490da58951) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..f8ee95523252ee7b59d52c411735c6490da58951) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927571413
@Dandandan is it possible this got stuck again on your tpcds run?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3929033989
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3941339937
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3941340119
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3941339937).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20417_3919093898.sh` | Dandandan | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919093898` |
| `20417_3919839904.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919839904` |
| `arrow-9118-3925061519.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9118#issuecomment-3925061519` |
| `20363_3927114392.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392` |
| `arrow-9441-3928995639.sh` | Dandandan | boolean_kernels | `https://github.com/apache/arrow-rs/pull/9441#issuecomment-3928995639` |
| `20444_3935317555.sh` | adriangb | in_list | `https://github.com/apache/datafusion/pull/20444#issuecomment-3935317555` |
| `20417_3939015858.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3939015858` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905692963
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3905692013).
No pending jobs in `jobs/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903539063
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905798461
run benchmark tpds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945793206
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945883002
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979989424
I think this PR gets OOM killed -- I'll remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977533074
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977531970
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979874362
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979876072
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960722763
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3961020612
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.56 ms β 2.63 ms β no change β
β QQuery 1 β 52.46 ms β 52.89 ms β no change β
β QQuery 2 β 131.74 ms β 130.76 ms β no change β
β QQuery 3 β 157.86 ms β 156.25 ms β no change β
β QQuery 4 β 995.97 ms β 1031.06 ms β no change β
β QQuery 5 β 1270.00 ms β 1264.02 ms β no change β
β QQuery 6 β 17.80 ms β 9.34 ms β +1.90x faster β
β QQuery 7 β 68.58 ms β 65.71 ms β no change β
β QQuery 8 β 1365.22 ms β 1415.75 ms β no change β
β QQuery 9 β 1776.90 ms β 1764.53 ms β no change β
β QQuery 10 β 479.01 ms β 483.21 ms β no change β
β QQuery 11 β 528.89 ms β 530.44 ms β no change β
β QQuery 12 β 1371.54 ms β 1384.82 ms β no change β
β QQuery 13 β 2018.88 ms β 2040.67 ms β no change β
β QQuery 14 β 1388.12 ms β 1403.30 ms β no change β
β QQuery 15 β 1146.12 ms β 1176.75 ms β no change β
β QQuery 16 β 2444.29 ms β 2529.31 ms β no change β
β QQuery 17 β 2430.13 ms β 2482.21 ms β no change β
β QQuery 18 β 5264.45 ms β 4906.84 ms β +1.07x faster β
β QQuery 19 β 135.54 ms β 137.71 ms β no change β
β QQuery 20 β 1740.32 ms β 1750.65 ms β no change β
β QQuery 21 β 2208.20 ms β 2144.18 ms β no change β
β QQuery 22 β 3728.19 ms β 3088.91 ms β +1.21x faster β
β QQuery 23 β 1055.02 ms β 1068.56 ms β no change β
β QQuery 24 β 243.01 ms β 212.54 ms β +1.14x faster β
β QQuery 25 β 599.75 ms β 608.94 ms β no change β
β QQuery 26 β 340.35 ms β 227.70 ms β +1.49x faster β
β QQuery 27 β 2769.48 ms β 2339.84 ms β +1.18x faster β
β QQuery 28 β 22177.26 ms β 23878.20 ms β 1.08x slower β
β QQuery 29 β 967.67 ms β 955.31 ms β no change β
β QQuery 30 β 1288.14 ms β 1231.92 ms β no change β
β QQuery 31 β 1341.31 ms β 1321.31 ms β no change β
β QQuery 32 β 4533.90 ms β 4026.53 ms β +1.13x faster β
β QQuery 33 β 5366.53 ms β 5044.04 ms β +1.06x faster β
β QQuery 34 β 5411.59 ms β 5346.41 ms β no change β
β QQuery 35 β 1833.12 ms β 1857.85 ms β no change β
β QQuery 36 β 177.99 ms β 159.91 ms β +1.11x faster β
β QQuery 37 β 89.31 ms β 72.87 ms β +1.23x faster β
β QQuery 38 β 87.03 ms β 91.05 ms β no change β
β QQuery 39 β 278.76 ms β 281.03 ms β no change β
β QQuery 40 β 56.86 ms β 48.66 ms β +1.17x faster β
β QQuery 41 β 49.16 ms β 33.22 ms β +1.48x faster β
β QQuery 42 β 35.87 ms β 52.40 ms β 1.46x slower β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 79424.85ms β
β Total Time (filter-pushdown-dynamic-bytes) β 78810.20ms β
β Average Time (HEAD) β 1847.09ms β
β Average Time (filter-pushdown-dynamic-bytes) β 1832.80ms β
β Queries Faster β 12 β
β Queries Slower β 2 β
β Queries with No Change β 29 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947785937
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3949585376
Yeah, seems like itβs always tpcds? I donβt think itβs this branch necessarily, it got stuck on your branch earlier and this one has been pretty much completely rewritten since last time it got stuck here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3956080042
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (aaa75f256caa1f3c66d9c91ae563f2b2635c563d) to b9328b9734534c8e2ce92b00c5368f88956bf5c3 [diff](https://github.com/apache/datafusion/compare/b9328b9734534c8e2ce92b00c5368f88956bf5c3..aaa75f256caa1f3c66d9c91ae563f2b2635c563d) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905145488
It might be time to invest in a more legit benchmark runner strategy -- my bash script nest is not super reliable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905692013
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905800281
run benchmark tpcds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229997526
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4229959251)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.22 / 4.51 Β±6.43 / 17.36 ms β 1.21 / 4.43 Β±6.31 / 17.04 ms β no change β
β QQuery 1 β 13.99 / 14.34 Β±0.21 / 14.57 ms β 13.99 / 14.42 Β±0.22 / 14.63 ms β no change β
β QQuery 2 β 42.55 / 43.09 Β±0.47 / 43.85 ms β 43.24 / 43.61 Β±0.35 / 44.20 ms β no change β
β QQuery 3 β 44.18 / 45.50 Β±1.19 / 47.12 ms β 44.48 / 45.89 Β±0.96 / 47.32 ms β no change β
β QQuery 4 β 285.82 / 293.40 Β±6.51 / 305.17 ms β 279.94 / 287.67 Β±6.73 / 295.65 ms β no change β
β QQuery 5 β 338.67 / 342.39 Β±3.36 / 346.74 ms β 333.40 / 342.81 Β±5.66 / 349.09 ms β no change β
β QQuery 6 β 5.82 / 6.62 Β±0.59 / 7.54 ms β 5.71 / 7.23 Β±0.95 / 8.20 ms β 1.09x slower β
β QQuery 7 β 16.64 / 17.43 Β±1.09 / 19.55 ms β 16.42 / 17.06 Β±0.82 / 18.67 ms β no change β
β QQuery 8 β 420.57 / 436.43 Β±10.20 / 451.25 ms β 410.84 / 419.75 Β±9.65 / 435.37 ms β no change β
β QQuery 9 β 664.81 / 667.72 Β±4.82 / 677.32 ms β 628.83 / 635.51 Β±4.65 / 641.64 ms β no change β
β QQuery 10 β 91.14 / 93.69 Β±2.00 / 96.83 ms β 91.41 / 93.14 Β±1.67 / 96.16 ms β no change β
β QQuery 11 β 107.05 / 108.12 Β±0.73 / 108.89 ms β 102.28 / 103.93 Β±1.05 / 104.85 ms β no change β
β QQuery 12 β 349.12 / 355.42 Β±3.82 / 358.67 ms β 332.95 / 341.98 Β±6.85 / 353.94 ms β no change β
β QQuery 13 β 458.42 / 469.26 Β±14.42 / 496.92 ms β 453.92 / 464.32 Β±10.10 / 479.28 ms β no change β
β QQuery 14 β 342.24 / 351.82 Β±5.67 / 358.08 ms β 339.75 / 346.08 Β±3.34 / 348.63 ms β no change β
β QQuery 15 β 354.54 / 365.98 Β±11.79 / 380.87 ms β 346.21 / 361.96 Β±14.39 / 380.13 ms β no change β
β QQuery 16 β 712.21 / 724.77 Β±10.28 / 740.00 ms β 703.77 / 717.27 Β±15.25 / 744.17 ms β no change β
β QQuery 17 β 710.31 / 742.64 Β±28.69 / 793.54 ms β 703.62 / 710.80 Β±5.28 / 717.01 ms β no change β
β QQuery 18 β 1399.72 / 1503.23 Β±60.57 / 1567.89 ms β 1390.74 / 1431.08 Β±32.71 / 1480.30 ms β no change β
β QQuery 19 β 37.33 / 40.07 Β±3.29 / 46.40 ms β 35.21 / 52.40 Β±32.19 / 116.70 ms β 1.31x slower β
β QQuery 20 β 721.43 / 743.33 Β±20.60 / 776.90 ms β 721.98 / 732.91 Β±13.08 / 755.68 ms β no change β
β QQuery 21 β 760.12 / 769.53 Β±5.64 / 776.88 ms β 760.55 / 772.54 Β±14.27 / 800.04 ms β no change β
β QQuery 22 β 1136.41 / 1143.04 Β±7.09 / 1155.96 ms β 1136.83 / 1140.80 Β±3.07 / 1145.73 ms β no change β
β QQuery 23 β 3076.68 / 3091.18 Β±8.20 / 3099.38 ms β 3063.61 / 3094.24 Β±17.34 / 3107.76 ms β no change β
β QQuery 24 β 99.73 / 101.29 Β±1.16 / 103.19 ms β 99.86 / 101.72 Β±1.93 / 105.12 ms β no change β
β QQuery 25 β 138.53 / 140.49 Β±1.56 / 143.00 ms β 136.52 / 140.44 Β±2.88 / 145.47 ms β no change β
β QQuery 26 β 97.54 / 102.55 Β±2.71 / 105.12 ms β 99.26 / 102.33 Β±2.19 / 105.56 ms β no change β
β QQuery 27 β 850.58 / 855.12 Β±5.39 / 865.10 ms β 841.50 / 855.30 Β±15.48 / 885.03 ms β no change β
β QQuery 28 β 3264.06 / 3297.98 Β±17.25 / 3312.26 ms β 3259.13 / 3285.74 Β±20.91 / 3322.51 ms β no change β
β QQuery 29 β 50.98 / 55.81 Β±4.80 / 61.96 ms β 49.34 / 54.69 Β±5.44 / 63.07 ms β no change β
β QQuery 30 β 362.29 / 368.58 Β±4.00 / 374.83 ms β 360.40 / 363.33 Β±3.65 / 370.50 ms β no change β
β QQuery 31 β 361.57 / 375.94 Β±11.68 / 390.72 ms β 344.95 / 367.79 Β±11.57 / 376.80 ms β no change β
β QQuery 32 β 1144.69 / 1288.62 Β±78.77 / 1373.76 ms β 1021.05 / 1034.94 Β±14.54 / 1062.23 ms β +1.25x faster β
β QQuery 33 β 1444.66 / 1459.43 Β±21.36 / 1501.74 ms β 1452.25 / 1488.52 Β±23.37 / 1519.66 ms β no change β
β QQuery 34 β 1436.75 / 1461.38 Β±20.91 / 1494.83 ms β 1462.53 / 1480.18 Β±10.94 / 1490.52 ms β no change β
β QQuery 35 β 376.88 / 382.04 Β±4.20 / 389.34 ms β 378.00 / 382.07 Β±3.67 / 388.62 ms β no change β
β QQuery 36 β 108.71 / 114.93 Β±4.16 / 120.24 ms β 111.50 / 117.80 Β±3.59 / 122.22 ms β no change β
β QQuery 37 β 46.60 / 48.17 Β±1.01 / 49.36 ms β 46.86 / 48.71 Β±1.36 / 50.97 ms β no change β
β QQuery 38 β 74.05 / 76.01 Β±1.46 / 78.49 ms β 73.03 / 74.31 Β±0.74 / 75.14 ms β no change β
β QQuery 39 β 197.19 / 208.37 Β±6.76 / 215.29 ms β 198.66 / 215.05 Β±9.37 / 226.01 ms β no change β
β QQuery 40 β 22.98 / 25.18 Β±2.70 / 30.26 ms β 21.78 / 24.02 Β±2.55 / 28.93 ms β no change β
β QQuery 41 β 19.23 / 20.17 Β±0.76 / 20.96 ms β 18.86 / 20.98 Β±1.38 / 23.19 ms β no change β
β QQuery 42 β 18.59 / 19.45 Β±0.44 / 19.78 ms β 19.28 / 19.88 Β±0.73 / 21.21 ms β no change β
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 22775.03ms β
β Total Time (filter-pushdown-dynamic-bytes) β 22359.63ms β
β Average Time (HEAD) β 529.65ms β
β Average Time (filter-pushdown-dynamic-bytes) β 519.99ms β
β Queries Faster β 1 β
β Queries Slower β 2 β
β Queries with No Change β 40 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 115.0s |
| Peak memory | 40.8 GiB |
| Avg memory | 30.2 GiB |
| CPU user | 1075.0s |
| CPU sys | 91.5s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 112.9s |
| Peak memory | 41.4 GiB |
| Avg memory | 30.2 GiB |
| CPU user | 1065.5s |
| CPU sys | 82.9s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903262333
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903262814
run benchmark tpch
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904317167
FYI @alamb
> Hm it seems stuck again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904489818
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3904489674).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3903261774.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903261774` |
| `20363_3903262814.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903262814` |
| `20365_3903537986.sh` | Dandandan | tpch tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903537986` |
| `20365_3903568877.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903568877` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905798773
π€ Hi @adriangb, thanks for the request (https://github.com/apache/datafusion/pull/20363#issuecomment-3905798461).
[`scrape_comments.py`](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/scrape_comments.py) only supports whitelisted benchmarks.
- Standard: clickbench_1, clickbench_extended, clickbench_partitioned, clickbench_pushdown, external_aggr, tpcds, tpch, tpch10, tpch_mem, tpch_mem10
- Criterion: aggregate_query_sql, aggregate_vectorized, case_when, character_length, in_list, left, plan_reuse, range_and_generate_series, replace, reset_plan_states, sort, sql_planner, strpos, substr_index, with_hashes
Please choose one or more of these with `run benchmark <name>` or `run benchmark <name1> <name2>...`
You can also set environment variables on subsequent lines:
```
run benchmark tpch_mem
DATAFUSION_RUNTIME_MEMORY_LIMIT=1G
```
Unsupported benchmarks: tpds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3905166961
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark tpch_sf1.json
--------------------
βββββββββββββ³ββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 1 β 177.77 ms β 178.75 ms β no change β
β QQuery 2 β 85.09 ms β 87.81 ms β no change β
β QQuery 3 β 123.49 ms β 124.67 ms β no change β
β QQuery 4 β 78.67 ms β 78.87 ms β no change β
β QQuery 5 β 167.07 ms β 171.53 ms β no change β
β QQuery 6 β 67.60 ms β 66.43 ms β no change β
β QQuery 7 β 203.17 ms β 210.59 ms β no change β
β QQuery 8 β 167.43 ms β 161.48 ms β no change β
β QQuery 9 β 219.47 ms β 226.33 ms β no change β
β QQuery 10 β 182.38 ms β 184.62 ms β no change β
β QQuery 11 β 61.54 ms β 61.02 ms β no change β
β QQuery 12 β 116.05 ms β 119.41 ms β no change β
β QQuery 13 β 219.46 ms β 224.87 ms β no change β
β QQuery 14 β 90.28 ms β 89.78 ms β no change β
β QQuery 15 β 124.49 ms β 126.86 ms β no change β
β QQuery 16 β 59.90 ms β 59.10 ms β no change β
β QQuery 17 β 260.50 ms β 257.03 ms β no change β
β QQuery 18 β 308.75 ms β 312.50 ms β no change β
β QQuery 19 β 129.27 ms β 132.84 ms β no change β
β QQuery 20 β 125.78 ms β 128.50 ms β no change β
β QQuery 21 β 252.25 ms β 258.00 ms β no change β
β QQuery 22 β 40.02 ms β 40.25 ms β no change β
βββββββββββββ΄ββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³ββββββββββββ
β Benchmark Summary β β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 3260.42ms β
β Total Time (filter-pushdown-dynamic-bytes) β 3301.25ms β
β Average Time (HEAD) β 148.20ms β
β Average Time (filter-pushdown-dynamic-bytes) β 150.06ms β
β Queries Faster β 0 β
β Queries Slower β 0 β
β Queries with No Change β 22 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927148524
benchmarkj queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230031788
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230007775)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.23 / 4.59 Β±6.51 / 17.60 ms β 1.21 / 4.51 Β±6.43 / 17.37 ms β no change β
β QQuery 1 β 15.63 / 16.14 Β±0.30 / 16.50 ms β 7.22 / 7.80 Β±0.33 / 8.11 ms β +2.07x faster β
β QQuery 2 β 43.67 / 44.04 Β±0.22 / 44.30 ms β 44.21 / 44.89 Β±0.48 / 45.61 ms β no change β
β QQuery 3 β 43.99 / 47.37 Β±2.56 / 50.48 ms β 42.21 / 45.98 Β±2.29 / 48.02 ms β no change β
β QQuery 4 β 320.77 / 335.20 Β±10.82 / 346.76 ms β 294.12 / 298.96 Β±4.00 / 305.59 ms β +1.12x faster β
β QQuery 5 β 370.09 / 385.33 Β±9.21 / 397.80 ms β 351.84 / 355.66 Β±3.18 / 359.52 ms β +1.08x faster β
β QQuery 6 β 6.42 / 7.18 Β±0.84 / 8.75 ms β 6.06 / 6.51 Β±0.37 / 7.04 ms β +1.10x faster β
β QQuery 7 β 22.69 / 23.72 Β±1.31 / 26.21 ms β 22.29 / 23.12 Β±0.92 / 24.85 ms β no change β
β QQuery 8 β 444.24 / 460.03 Β±10.45 / 473.82 ms β 423.32 / 435.14 Β±6.58 / 441.82 ms β +1.06x faster β
β QQuery 9 β 712.27 / 732.71 Β±20.01 / 767.34 ms β 651.08 / 664.98 Β±8.85 / 676.14 ms β +1.10x faster β
β QQuery 10 β 122.41 / 128.04 Β±3.97 / 132.54 ms β 117.90 / 119.95 Β±2.26 / 124.09 ms β +1.07x faster β
β QQuery 11 β 135.18 / 141.23 Β±4.96 / 147.77 ms β 130.18 / 132.33 Β±1.54 / 134.86 ms β +1.07x faster β
β QQuery 12 β 396.41 / 402.47 Β±6.46 / 412.55 ms β 375.89 / 378.32 Β±2.83 / 383.56 ms β +1.06x faster β
β QQuery 13 β 512.28 / 530.28 Β±17.63 / 560.37 ms β 477.83 / 490.80 Β±12.25 / 507.95 ms β +1.08x faster β
β QQuery 14 β 406.23 / 411.22 Β±5.90 / 422.30 ms β 377.42 / 384.12 Β±4.38 / 390.25 ms β +1.07x faster β
β QQuery 15 β 391.84 / 405.86 Β±14.42 / 428.66 ms β 352.46 / 369.17 Β±20.59 / 407.82 ms β +1.10x faster β
β QQuery 16 β 739.13 / 751.75 Β±15.37 / 781.70 ms β 698.75 / 709.55 Β±8.11 / 720.04 ms β +1.06x faster β
β QQuery 17 β 751.81 / 759.98 Β±4.93 / 767.00 ms β 698.22 / 708.34 Β±5.20 / 713.04 ms β +1.07x faster β
β QQuery 18 β 1520.48 / 1546.22 Β±23.29 / 1582.14 ms β 1438.45 / 1460.57 Β±29.45 / 1517.00 ms β +1.06x faster β
β QQuery 19 β 38.02 / 39.35 Β±1.38 / 41.96 ms β 36.37 / 45.18 Β±14.98 / 75.05 ms β 1.15x slower β
β QQuery 20 β 723.45 / 743.49 Β±19.08 / 769.07 ms β 6.57 / 6.92 Β±0.51 / 7.93 ms β +107.47x faster β
β QQuery 21 β 711.53 / 729.48 Β±13.61 / 753.19 ms β 738.58 / 764.68 Β±16.84 / 790.78 ms β no change β
β QQuery 22 β 1157.77 / 1170.00 Β±9.50 / 1181.57 ms β 901.43 / 924.30 Β±21.80 / 956.28 ms β +1.27x faster β
β QQuery 23 β 261.59 / 264.54 Β±2.46 / 267.45 ms β 318.35 / 614.12 Β±222.16 / 817.10 ms β 2.32x slower β
β QQuery 24 β 90.09 / 93.82 Β±2.98 / 99.08 ms β 68.06 / 78.68 Β±6.51 / 86.52 ms β +1.19x faster β
β QQuery 25 β 172.38 / 178.21 Β±5.14 / 187.01 ms β 175.05 / 180.16 Β±4.50 / 185.96 ms β no change β
β QQuery 26 β 111.71 / 116.31 Β±2.71 / 119.95 ms β 99.60 / 101.80 Β±1.82 / 104.08 ms β +1.14x faster β
β QQuery 27 β 973.15 / 984.74 Β±8.76 / 996.63 ms β 832.45 / 839.61 Β±8.22 / 855.06 ms β +1.17x faster β
β QQuery 28 β 3387.80 / 3441.88 Β±29.26 / 3470.20 ms β 3338.87 / 3398.40 Β±42.30 / 3467.58 ms β no change β
β QQuery 29 β 49.57 / 55.68 Β±5.20 / 63.87 ms β 51.20 / 56.61 Β±4.96 / 65.53 ms β no change β
β QQuery 30 β 370.26 / 374.58 Β±3.99 / 380.72 ms β 758.67 / 770.15 Β±6.25 / 777.65 ms β 2.06x slower β
β QQuery 31 β 357.36 / 372.30 Β±8.08 / 380.26 ms β 380.25 / 1138.83 Β±409.13 / 1606.32 ms β 3.06x slower β
β QQuery 32 β 1333.35 / 1371.82 Β±20.21 / 1391.67 ms β 1041.69 / 1212.19 Β±113.08 / 1322.04 ms β +1.13x faster β
β QQuery 33 β 1493.86 / 1533.72 Β±32.22 / 1592.46 ms β 1447.01 / 1465.66 Β±16.19 / 1489.25 ms β no change β
β QQuery 34 β 1486.53 / 1499.60 Β±12.59 / 1523.38 ms β 1462.02 / 1469.83 Β±7.43 / 1483.06 ms β no change β
β QQuery 35 β 377.85 / 385.28 Β±5.25 / 392.09 ms β 393.11 / 395.59 Β±1.52 / 397.51 ms β no change β
β QQuery 36 β 102.17 / 107.96 Β±3.61 / 112.36 ms β 134.19 / 145.62 Β±10.51 / 161.47 ms β 1.35x slower β
β QQuery 37 β 53.44 / 57.13 Β±2.91 / 62.39 ms β 47.01 / 50.34 Β±2.48 / 54.13 ms β +1.13x faster β
β QQuery 38 β 56.12 / 59.39 Β±2.77 / 63.12 ms β 55.86 / 90.18 Β±40.00 / 157.19 ms β 1.52x slower β
β QQuery 39 β 161.95 / 173.49 Β±6.65 / 181.42 ms β 236.99 / 246.45 Β±8.58 / 260.38 ms β 1.42x slower β
β QQuery 40 β 30.82 / 32.50 Β±1.52 / 35.33 ms β 45.01 / 47.14 Β±1.96 / 50.18 ms β 1.45x slower β
β QQuery 41 β 26.19 / 27.01 Β±0.70 / 28.08 ms β 24.38 / 26.74 Β±1.88 / 30.01 ms β no change β
β QQuery 42 β 21.53 / 21.97 Β±0.38 / 22.54 ms β 21.50 / 22.15 Β±0.72 / 23.48 ms β no change β
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 20967.57ms β
β Total Time (filter-pushdown-dynamic-bytes) β 20732.00ms β
β Average Time (HEAD) β 487.62ms β
β Average Time (filter-pushdown-dynamic-bytes) β 482.14ms β
β Queries Faster β 22 β
β Queries Slower β 8 β
β Queries with No Change β 13 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 106.2s |
| Peak memory | 41.3 GiB |
| Avg memory | 29.1 GiB |
| CPU user | 949.5s |
| CPU sys | 102.4s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 104.8s |
| Peak memory | 40.0 GiB |
| Avg memory | 29.0 GiB |
| CPU user | 939.7s |
| CPU sys | 106.5s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230283640
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230259802)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.19 / 4.41 Β±6.33 / 17.08 ms β 1.22 / 4.46 Β±6.36 / 17.18 ms β no change β
β QQuery 1 β 14.38 / 14.58 Β±0.15 / 14.81 ms β 14.33 / 14.69 Β±0.20 / 14.91 ms β no change β
β QQuery 2 β 42.82 / 43.37 Β±0.34 / 43.83 ms β 43.55 / 44.22 Β±0.37 / 44.66 ms β no change β
β QQuery 3 β 40.51 / 43.63 Β±3.10 / 48.70 ms β 44.41 / 46.39 Β±1.05 / 47.42 ms β 1.06x slower β
β QQuery 4 β 291.23 / 298.31 Β±5.26 / 306.06 ms β 286.25 / 294.08 Β±4.69 / 299.87 ms β no change β
β QQuery 5 β 344.45 / 348.29 Β±2.83 / 352.50 ms β 340.63 / 345.58 Β±3.15 / 348.94 ms β no change β
β QQuery 6 β 5.01 / 5.55 Β±0.41 / 6.26 ms β 6.22 / 6.78 Β±1.04 / 8.85 ms β 1.22x slower β
β QQuery 7 β 16.74 / 17.48 Β±0.53 / 18.17 ms β 16.84 / 17.58 Β±0.45 / 18.01 ms β no change β
β QQuery 8 β 414.59 / 427.02 Β±7.89 / 436.27 ms β 423.68 / 429.21 Β±3.00 / 432.78 ms β no change β
β QQuery 9 β 654.25 / 658.50 Β±2.69 / 662.58 ms β 646.47 / 651.10 Β±2.83 / 655.17 ms β no change β
β QQuery 10 β 91.66 / 94.69 Β±3.27 / 100.74 ms β 90.56 / 92.81 Β±2.42 / 97.10 ms β no change β
β QQuery 11 β 103.27 / 104.20 Β±0.87 / 105.82 ms β 102.93 / 104.60 Β±0.95 / 105.52 ms β no change β
β QQuery 12 β 334.80 / 342.88 Β±5.34 / 349.51 ms β 340.25 / 345.06 Β±3.75 / 350.94 ms β no change β
β QQuery 13 β 457.12 / 467.29 Β±8.86 / 483.08 ms β 446.91 / 473.39 Β±19.74 / 506.43 ms β no change β
β QQuery 14 β 347.95 / 351.47 Β±2.68 / 355.57 ms β 344.49 / 346.23 Β±1.76 / 349.39 ms β no change β
β QQuery 15 β 348.49 / 371.82 Β±15.33 / 393.33 ms β 346.69 / 360.96 Β±8.84 / 370.89 ms β no change β
β QQuery 16 β 715.80 / 743.08 Β±25.15 / 786.01 ms β 719.22 / 729.99 Β±6.94 / 740.08 ms β no change β
β QQuery 17 β 713.65 / 720.00 Β±6.89 / 732.60 ms β 713.89 / 717.74 Β±3.79 / 724.15 ms β no change β
β QQuery 18 β 1464.24 / 1482.84 Β±14.48 / 1508.20 ms β 1373.49 / 1452.19 Β±44.67 / 1512.56 ms β no change β
β QQuery 19 β 35.76 / 40.30 Β±7.64 / 55.54 ms β 34.89 / 36.37 Β±1.59 / 38.42 ms β +1.11x faster β
β QQuery 20 β 721.50 / 739.22 Β±18.16 / 765.72 ms β 706.69 / 716.71 Β±10.44 / 735.73 ms β no change β
β QQuery 21 β 769.23 / 778.25 Β±9.25 / 789.86 ms β 756.05 / 763.87 Β±5.18 / 771.82 ms β no change β
β QQuery 22 β 1141.56 / 1144.48 Β±3.91 / 1152.13 ms β 1130.23 / 1133.58 Β±2.92 / 1138.87 ms β no change β
β QQuery 23 β 3090.97 / 3113.58 Β±22.99 / 3153.17 ms β 3062.89 / 3081.66 Β±17.28 / 3108.68 ms β no change β
β QQuery 24 β 98.90 / 101.75 Β±1.87 / 104.56 ms β 98.72 / 102.57 Β±2.64 / 106.27 ms β no change β
β QQuery 25 β 138.75 / 140.70 Β±1.53 / 142.45 ms β 138.59 / 141.51 Β±2.11 / 144.46 ms β no change β
β QQuery 26 β 97.61 / 102.12 Β±2.91 / 105.94 ms β 101.07 / 102.81 Β±1.53 / 105.30 ms β no change β
β QQuery 27 β 852.97 / 858.30 Β±6.68 / 871.04 ms β 846.47 / 853.63 Β±6.38 / 863.98 ms β no change β
β QQuery 28 β 3274.64 / 3297.34 Β±17.81 / 3316.93 ms β 3291.86 / 3323.03 Β±18.11 / 3341.74 ms β no change β
β QQuery 29 β 48.95 / 54.28 Β±5.34 / 63.63 ms β 50.77 / 56.86 Β±8.08 / 72.48 ms β no change β
β QQuery 30 β 360.85 / 368.38 Β±3.80 / 370.99 ms β 355.02 / 360.78 Β±5.15 / 368.08 ms β no change β
β QQuery 31 β 367.44 / 382.03 Β±11.55 / 393.53 ms β 366.33 / 377.65 Β±5.80 / 382.83 ms β no change β
β QQuery 32 β 1056.37 / 1072.57 Β±23.66 / 1119.05 ms β 1138.61 / 1282.17 Β±96.69 / 1440.00 ms β 1.20x slower β
β QQuery 33 β 1474.26 / 1485.47 Β±11.37 / 1506.57 ms β 1486.44 / 1515.70 Β±38.01 / 1590.30 ms β no change β
β QQuery 34 β 1450.02 / 1481.44 Β±23.04 / 1513.41 ms β 1445.49 / 1463.25 Β±10.68 / 1473.78 ms β no change β
β QQuery 35 β 398.40 / 401.67 Β±1.85 / 404.10 ms β 390.99 / 394.54 Β±2.90 / 398.85 ms β no change β
β QQuery 36 β 117.43 / 122.07 Β±3.95 / 127.25 ms β 113.44 / 121.18 Β±4.06 / 125.23 ms β no change β
β QQuery 37 β 48.49 / 50.27 Β±1.44 / 52.56 ms β 48.15 / 49.86 Β±1.27 / 51.56 ms β no change β
β QQuery 38 β 75.81 / 77.49 Β±1.60 / 80.21 ms β 75.25 / 77.46 Β±1.64 / 79.66 ms β no change β
β QQuery 39 β 213.13 / 223.68 Β±7.68 / 232.78 ms β 216.79 / 222.60 Β±5.08 / 230.80 ms β no change β
β QQuery 40 β 24.87 / 25.60 Β±0.93 / 27.20 ms β 22.26 / 25.87 Β±2.32 / 28.76 ms β no change β
β QQuery 41 β 19.10 / 20.96 Β±1.58 / 23.89 ms β 20.03 / 22.16 Β±1.95 / 25.26 ms β 1.06x slower β
β QQuery 42 β 19.26 / 19.92 Β±0.62 / 20.81 ms β 19.59 / 20.21 Β±0.49 / 20.78 ms β no change β
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 22641.32ms β
β Total Time (filter-pushdown-dynamic-bytes) β 22723.07ms β
β Average Time (HEAD) β 526.54ms β
β Average Time (filter-pushdown-dynamic-bytes) β 528.44ms β
β Queries Faster β 1 β
β Queries Slower β 4 β
β Queries with No Change β 38 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 114.3s |
| Peak memory | 38.1 GiB |
| Avg memory | 28.5 GiB |
| CPU user | 1078.7s |
| CPU sys | 84.1s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 114.8s |
| Peak memory | 42.1 GiB |
| Avg memory | 32.1 GiB |
| CPU user | 1074.7s |
| CPU sys | 93.6s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229961271
run benchmark clickbench_partitioned
```yaml
env:
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS: true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS: true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945883364
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3945883002).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20481_3943833990.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20481#issuecomment-3943833990` |
| `19728_3944810475.sh` | alamb | default | `https://github.com/apache/datafusion/pull/19728#issuecomment-3944810475` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3949581147
I think it is stuck again π
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947180599
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3976778015
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986
run benchmark tpcds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979871515
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979873710
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979872174
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979875753
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3980039525
> > I think this PR gets OOM killed -- I'll remove it
>
> Thanks. Sorry for triggering it again. Itβs hard to debug whatβs going on and Iβm surprised this PR causes an OOM when others donβt, but it does seem to be especially problematic. Wonder if it has a mem leak or something.
For context - I tried to run the Clickbench benchmark on a Claude Cloud runner (21 GB RAM / 16 cores) - there it also get stuck (without any changes).
So I guess some of the queries consume already a high amount of RAM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230259802
run benchmark clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927267807
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3927266957).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20417_3919093898.sh` | Dandandan | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919093898` |
| `20417_3919839904.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919839904` |
| `arrow-9118-3925061519.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9118#issuecomment-3925061519` |
| `20363_3927114392.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927266957
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945514669
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3945514401).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20444_3935317555.sh` | adriangb | in_list | `https://github.com/apache/datafusion/pull/20444#issuecomment-3935317555` |
| `20417_3939015858.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3939015858` |
| `20481_3943833990.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20481#issuecomment-3943833990` |
| `19728_3944810475.sh` | alamb | default | `https://github.com/apache/datafusion/pull/19728#issuecomment-3944810475` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903341081
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.52 ms β 2.78 ms β 1.10x slower β
β QQuery 1 β 52.24 ms β 49.61 ms β +1.05x faster β
β QQuery 2 β 132.32 ms β 136.95 ms β no change β
β QQuery 3 β 160.45 ms β 159.43 ms β no change β
β QQuery 4 β 1006.07 ms β 1001.72 ms β no change β
β QQuery 5 β 1264.63 ms β 1240.53 ms β no change β
β QQuery 6 β 17.57 ms β 6.46 ms β +2.72x faster β
β QQuery 7 β 68.54 ms β 55.34 ms β +1.24x faster β
β QQuery 8 β 1432.88 ms β 1357.09 ms β +1.06x faster β
β QQuery 9 β 1783.22 ms β 1723.13 ms β no change β
β QQuery 10 β 469.85 ms β 339.66 ms β +1.38x faster β
β QQuery 11 β 515.70 ms β 392.95 ms β +1.31x faster β
β QQuery 12 β 1424.70 ms β 1142.60 ms β +1.25x faster β
β QQuery 13 β 2086.66 ms β 1758.57 ms β +1.19x faster β
β QQuery 14 β 1465.46 ms β 1166.84 ms β +1.26x faster β
β QQuery 15 β 1205.75 ms β 1142.22 ms β +1.06x faster β
β QQuery 16 β 2455.92 ms β 2414.92 ms β no change β
β QQuery 17 β 2453.39 ms β 2375.62 ms β no change β
β QQuery 18 β 4799.30 ms β 4693.76 ms β no change β
β QQuery 19 β 141.25 ms β 142.58 ms β no change β
β QQuery 20 β 1861.44 ms β 1845.91 ms β no change β
β QQuery 21 β 2312.86 ms β 2179.66 ms β +1.06x faster β
β QQuery 22 β 3956.56 ms β 4079.52 ms β no change β
β QQuery 23 β 1067.61 ms β 4758.55 ms β 4.46x slower β
β QQuery 24 β 244.71 ms β 185.57 ms β +1.32x faster β
β QQuery 25 β 635.55 ms β 447.91 ms β +1.42x faster β
β QQuery 26 β 318.23 ms β 204.37 ms β +1.56x faster β
β QQuery 27 β 2945.79 ms β 2432.80 ms β +1.21x faster β
β QQuery 28 β 23684.76 ms β 23006.72 ms β no change β
β QQuery 29 β 948.88 ms β 986.96 ms β no change β
β QQuery 30 β 1269.24 ms β 1229.75 ms β no change β
β QQuery 31 β 1311.55 ms β 1343.10 ms β no change β
β QQuery 32 β 4161.16 ms β 3919.03 ms β +1.06x faster β
β QQuery 33 β 5017.59 ms β 5183.95 ms β no change β
β QQuery 34 β 5507.17 ms β 5238.45 ms β no change β
β QQuery 35 β 1862.72 ms β 1797.80 ms β no change β
β QQuery 36 β 171.63 ms β 185.39 ms β 1.08x slower β
β QQuery 37 β 90.51 ms β 72.77 ms β +1.24x faster β
β QQuery 38 β 85.76 ms β 111.77 ms β 1.30x slower β
β QQuery 39 β 286.54 ms β 332.16 ms β 1.16x slower β
β QQuery 40 β 56.28 ms β 39.13 ms β +1.44x faster β
β QQuery 41 β 50.29 ms β 35.28 ms β +1.43x faster β
β QQuery 42 β 36.46 ms β 32.72 ms β +1.11x faster β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 80821.72ms β
β Total Time (filter-pushdown-dynamic-bytes) β 80952.02ms β
β Average Time (HEAD) β 1879.57ms β
β Average Time (filter-pushdown-dynamic-bytes) β 1882.61ms β
β Queries Faster β 20 β
β Queries Slower β 5 β
β Queries with No Change β 18 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3904383525
@Dandandan this is mostly vibe coded, I'm only 50% confident it even makes sense without reviewing the code fwiw
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903261774
run benchmark tpcds
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903918963
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3903918629).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3903261774.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903261774` |
| `20363_3903262814.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903262814` |
| `20365_3903537986.sh` | Dandandan | tpch tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903537986` |
| `20365_3903568877.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903568877` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947181655
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (26111e94513e9c5c92d2058f5e33033361669c14) to 9660c9874315354ff22245699785f5f77841be80 [diff](https://github.com/apache/datafusion/compare/9660c9874315354ff22245699785f5f77841be80..26111e94513e9c5c92d2058f5e33033361669c14) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960343634
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3960343455).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `19728_3959360119.sh` | alamb | clickbench_partitioned | `https://github.com/apache/datafusion/pull/19728#issuecomment-3959360119` |
| `19728_3959359739.sh` | alamb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/19728#issuecomment-3959359739` |
| `20363_3959693326.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3959693326` |
| `20481_3960310838.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3960310838` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960624839
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββ³ββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 2.53 ms β 2.64 ms β no change β
β QQuery 1 β 52.35 ms β 52.96 ms β no change β
β QQuery 2 β 131.00 ms β 134.25 ms β no change β
β QQuery 3 β 154.14 ms β 151.73 ms β no change β
β QQuery 4 β 1010.08 ms β 1014.38 ms β no change β
β QQuery 5 β 1238.25 ms β 1262.65 ms β no change β
β QQuery 6 β 17.37 ms β 7.24 ms β +2.40x faster β
β QQuery 7 β 68.70 ms β 68.96 ms β no change β
β QQuery 8 β 1358.20 ms β 1408.47 ms β no change β
β QQuery 9 β 1734.13 ms β 1763.40 ms β no change β
β QQuery 10 β 482.46 ms β 487.49 ms β no change β
β QQuery 11 β 534.68 ms β 546.47 ms β no change β
β QQuery 12 β 1376.64 ms β 1400.04 ms β no change β
β QQuery 13 β 2090.56 ms β 2049.19 ms β no change β
β QQuery 14 β 1392.07 ms β 1405.61 ms β no change β
β QQuery 15 β 1153.69 ms β 1188.94 ms β no change β
β QQuery 16 β 2420.32 ms β 2468.34 ms β no change β
β QQuery 17 β 2439.14 ms β 2475.68 ms β no change β
β QQuery 18 β 4696.48 ms β 4827.59 ms β no change β
β QQuery 19 β 138.08 ms β 140.65 ms β no change β
β QQuery 20 β 1770.09 ms β 1777.28 ms β no change β
β QQuery 21 β 2183.88 ms β 2157.75 ms β no change β
β QQuery 22 β 3747.64 ms β 2942.05 ms β +1.27x faster β
β QQuery 23 β 1045.27 ms β 1207.91 ms β 1.16x slower β
β QQuery 24 β 241.92 ms β 204.22 ms β +1.18x faster β
β QQuery 25 β 608.39 ms β 628.69 ms β no change β
β QQuery 26 β 336.69 ms β 238.10 ms β +1.41x faster β
β QQuery 27 β 2799.20 ms β 2438.20 ms β +1.15x faster β
β QQuery 28 β 21908.37 ms β 23904.20 ms β 1.09x slower β
β QQuery 29 β 998.99 ms β 952.53 ms β no change β
β QQuery 30 β 1265.27 ms β 1272.97 ms β no change β
β QQuery 31 β 1311.55 ms β 1304.77 ms β no change β
β QQuery 32 β 3951.87 ms β 4131.34 ms β no change β
β QQuery 33 β 4967.33 ms β 5084.78 ms β no change β
β QQuery 34 β 5583.07 ms β 5855.00 ms β no change β
β QQuery 35 β 1842.97 ms β 1874.90 ms β no change β
β QQuery 36 β 181.72 ms β 168.30 ms β +1.08x faster β
β QQuery 37 β 86.77 ms β 86.91 ms β no change β
β QQuery 38 β 84.89 ms β 94.07 ms β 1.11x slower β
β QQuery 39 β 289.83 ms β 291.41 ms β no change β
β QQuery 40 β 59.72 ms β 54.82 ms β +1.09x faster β
β QQuery 41 β 51.56 ms β 56.45 ms β 1.09x slower β
β QQuery 42 β 38.89 ms β 52.88 ms β 1.36x slower β
βββββββββββββ΄ββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 77846.73ms β
β Total Time (filter-pushdown-dynamic-bytes) β 79636.21ms β
β Average Time (HEAD) β 1810.39ms β
β Average Time (filter-pushdown-dynamic-bytes) β 1852.00ms β
β Queries Faster β 7 β
β Queries Slower β 5 β
β Queries with No Change β 31 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3959693326
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979889287
> @Dandandan i deleted your comment to stop the spam. I honestly donβt know whatβs wrong with this PR or the runner. I think I should close it and open a new one.
Thanks... Perhaps it has to do with the comment / commit window?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3980006672
> I think this PR gets OOM killed -- I'll remove it
Thanks. Sorry for triggering it again. Itβs hard to debug whatβs going on and Iβm surprised this PR causes an OOM when others donβt, but it does seem to be especially problematic. Wonder if it has a mem leak or something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3976548888
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
DATAFUSION_EXECUTION_TARGET_PARTITIONS=1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977531736
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977534716
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977532761
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230282365
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230259644)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³βββββββββββββββββββββββββββββββββββββββββ³βββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.20 / 4.40 Β±6.32 / 17.03 ms β 1.19 / 4.48 Β±6.43 / 17.34 ms β no change β
β QQuery 1 β 15.26 / 15.61 Β±0.19 / 15.84 ms β 15.25 / 15.50 Β±0.20 / 15.80 ms β no change β
β QQuery 2 β 43.40 / 43.67 Β±0.21 / 43.95 ms β 44.30 / 44.69 Β±0.23 / 45.00 ms β no change β
β QQuery 3 β 43.13 / 44.92 Β±1.04 / 46.20 ms β 44.84 / 45.58 Β±0.86 / 47.03 ms β no change β
β QQuery 4 β 280.50 / 284.22 Β±3.81 / 291.33 ms β 284.32 / 292.02 Β±6.19 / 300.20 ms β no change β
β QQuery 5 β 337.78 / 340.47 Β±2.59 / 344.50 ms β 335.18 / 340.36 Β±3.40 / 344.10 ms β no change β
β QQuery 6 β 5.81 / 8.02 Β±2.24 / 11.97 ms β 6.55 / 7.99 Β±1.62 / 11.01 ms β no change β
β QQuery 7 β 21.66 / 22.23 Β±0.50 / 23.08 ms β 17.88 / 18.49 Β±0.60 / 19.24 ms β +1.20x faster β
β QQuery 8 β 398.40 / 413.49 Β±9.74 / 427.56 ms β 420.41 / 426.35 Β±4.32 / 431.77 ms β no change β
β QQuery 9 β 631.02 / 632.32 Β±1.20 / 634.35 ms β 640.52 / 648.38 Β±5.50 / 656.86 ms β no change β
β QQuery 10 β 117.27 / 120.04 Β±2.65 / 124.91 ms β 117.76 / 119.51 Β±1.85 / 122.66 ms β no change β
β QQuery 11 β 130.16 / 131.18 Β±1.03 / 132.82 ms β 131.42 / 133.28 Β±1.48 / 135.25 ms β no change β
β QQuery 12 β 364.71 / 367.07 Β±2.63 / 371.32 ms β 337.55 / 344.28 Β±7.00 / 355.44 ms β +1.07x faster β
β QQuery 13 β 476.83 / 493.77 Β±13.26 / 517.17 ms β 471.11 / 487.73 Β±13.75 / 508.91 ms β no change β
β QQuery 14 β 375.86 / 381.59 Β±3.63 / 386.66 ms β 377.08 / 386.62 Β±6.14 / 394.42 ms β no change β
β QQuery 15 β 341.48 / 356.50 Β±11.92 / 372.21 ms β 348.62 / 370.99 Β±23.03 / 406.22 ms β no change β
β QQuery 16 β 708.25 / 720.52 Β±16.91 / 751.69 ms β 716.30 / 748.74 Β±21.59 / 776.50 ms β no change β
β QQuery 17 β 704.86 / 730.38 Β±33.03 / 793.79 ms β 742.54 / 803.80 Β±34.20 / 846.88 ms β 1.10x slower β
β QQuery 18 β 1428.40 / 1471.53 Β±44.70 / 1547.37 ms β 1521.86 / 1640.36 Β±59.86 / 1680.62 ms β 1.11x slower β
β QQuery 19 β 35.41 / 47.99 Β±21.33 / 90.59 ms β 37.64 / 39.73 Β±1.29 / 41.17 ms β +1.21x faster β
β QQuery 20 β 718.33 / 739.90 Β±17.46 / 758.38 ms β 731.50 / 746.87 Β±20.71 / 786.56 ms β no change β
β QQuery 21 β 734.87 / 753.30 Β±20.34 / 791.73 ms β 759.04 / 775.67 Β±9.74 / 788.48 ms β no change β
β QQuery 22 β 1136.67 / 1165.06 Β±15.41 / 1182.14 ms β 921.43 / 926.84 Β±4.59 / 934.42 ms β +1.26x faster β
β QQuery 23 β 259.34 / 266.56 Β±4.26 / 271.53 ms β 279.48 / 339.25 Β±33.32 / 374.43 ms β 1.27x slower β
β QQuery 24 β 90.70 / 93.91 Β±3.08 / 98.57 ms β 70.44 / 75.03 Β±3.74 / 80.27 ms β +1.25x faster β
β QQuery 25 β 174.13 / 177.53 Β±5.43 / 188.32 ms β 146.38 / 149.47 Β±1.93 / 152.02 ms β +1.19x faster β
β QQuery 26 β 109.83 / 112.41 Β±2.47 / 116.85 ms β 108.65 / 111.20 Β±2.11 / 113.72 ms β no change β
β QQuery 27 β 972.69 / 990.14 Β±12.55 / 1010.45 ms β 841.76 / 853.41 Β±13.28 / 879.38 ms β +1.16x faster β
β QQuery 28 β 3349.49 / 3424.08 Β±38.17 / 3457.40 ms β 3336.21 / 3361.97 Β±15.49 / 3380.08 ms β no change β
β QQuery 29 β 49.31 / 90.71 Β±48.97 / 164.56 ms β 51.81 / 56.62 Β±5.29 / 66.80 ms β +1.60x faster β
β QQuery 30 β 353.72 / 364.72 Β±10.67 / 383.87 ms β 377.15 / 385.97 Β±10.15 / 404.25 ms β 1.06x slower β
β QQuery 31 β 361.16 / 385.50 Β±17.43 / 409.50 ms β 359.40 / 374.58 Β±11.23 / 387.58 ms β no change β
β QQuery 32 β 1182.15 / 1271.36 Β±65.85 / 1351.65 ms β 1140.73 / 1294.83 Β±110.12 / 1484.54 ms β no change β
β QQuery 33 β 1602.25 / 1739.86 Β±115.08 / 1890.83 ms β 1481.52 / 1522.82 Β±40.97 / 1600.31 ms β +1.14x faster β
β QQuery 34 β 1517.03 / 1542.80 Β±34.52 / 1610.50 ms β 1451.48 / 1467.75 Β±10.99 / 1482.60 ms β no change β
β QQuery 35 β 382.49 / 391.23 Β±5.38 / 398.06 ms β 386.21 / 395.91 Β±5.96 / 402.53 ms β no change β
β QQuery 36 β 100.59 / 107.74 Β±4.32 / 113.43 ms β 100.64 / 102.43 Β±1.60 / 104.91 ms β no change β
β QQuery 37 β 50.59 / 56.13 Β±4.13 / 62.37 ms β 45.99 / 49.34 Β±2.88 / 53.67 ms β +1.14x faster β
β QQuery 38 β 53.39 / 57.90 Β±2.47 / 60.19 ms β 53.70 / 55.87 Β±1.96 / 58.52 ms β no change β
β QQuery 39 β 160.80 / 180.90 Β±11.30 / 195.49 ms β 172.09 / 181.59 Β±6.00 / 189.52 ms β no change β
β QQuery 40 β 30.55 / 31.81 Β±1.45 / 34.49 ms β 28.91 / 30.63 Β±1.53 / 32.60 ms β no change β
β QQuery 41 β 26.77 / 27.19 Β±0.43 / 27.91 ms β 23.58 / 25.29 Β±1.44 / 27.65 ms β +1.08x faster β
β QQuery 42 β 21.96 / 22.85 Β±0.93 / 24.36 ms β 22.29 / 24.13 Β±1.91 / 27.64 ms β 1.06x slower β
βββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 20623.50ms β
β Total Time (filter-pushdown-dynamic-bytes) β 20226.32ms β
β Average Time (HEAD) β 479.62ms β
β Average Time (filter-pushdown-dynamic-bytes) β 470.38ms β
β Queries Faster β 11 β
β Queries Slower β 5 β
β Queries with No Change β 27 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 104.4s |
| Peak memory | 42.0 GiB |
| Avg memory | 28.9 GiB |
| CPU user | 923.4s |
| CPU sys | 109.7s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 102.3s |
| Peak memory | 41.3 GiB |
| Avg memory | 30.5 GiB |
| CPU user | 925.1s |
| CPU sys | 100.2s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229964175
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4229961271)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4229961271-1093-7mgz6 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (f8ee95523252ee7b59d52c411735c6490da58951) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..f8ee95523252ee7b59d52c411735c6490da58951) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230263316
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230259644)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4230259644-1096-22fwk 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (72f296f766dbbc2a81c64b8c19f49ec73a39f96d) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..72f296f766dbbc2a81c64b8c19f49ec73a39f96d) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903262610
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (dbab02b688dadd45a70d6f2b4ec47893f5afc279) to 53b0ffb93d5fb662506439b6f1ffc34f2d7684ec [diff](https://github.com/apache/datafusion/compare/53b0ffb93d5fb662506439b6f1ffc34f2d7684ec..dbab02b688dadd45a70d6f2b4ec47893f5afc279) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903341110
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (dbab02b688dadd45a70d6f2b4ec47893f5afc279) to 53b0ffb93d5fb662506439b6f1ffc34f2d7684ec [diff](https://github.com/apache/datafusion/compare/53b0ffb93d5fb662506439b6f1ffc34f2d7684ec..dbab02b688dadd45a70d6f2b4ec47893f5afc279) using: tpcds
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3903539889
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3903539063).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3903261774.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903261774` |
| `20363_3903262814.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3903262814` |
| `20365_3903537986.sh` | Dandandan | tpch tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20365#issuecomment-3903537986` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927158556
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3927157700).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20417_3919093898.sh` | Dandandan | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919093898` |
| `20417_3919839904.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20417#issuecomment-3919839904` |
| `arrow-9118-3925061519.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9118#issuecomment-3925061519` |
| `20363_3927114392.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3927114392` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3927566224
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3949584191
@alamb could you take a look? Somehow the result is also empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3955645656
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3955645564).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20534_3955110156.sh` | alamb | sql_planner | `https://github.com/apache/datafusion/pull/20534#issuecomment-3955110156` |
| `20481_3955122956.sh` | alamb | default | `https://github.com/apache/datafusion/pull/20481#issuecomment-3955122956` |
| `20481_3955149024.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3955149024` |
| `20363_3955225206.sh` | adriangb | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3955225206` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3945470632
π€: Benchmark completed
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
```
</p>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947786469
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3947785937).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3947413716.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3947413716` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230011365
π€ Benchmark running (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230007775)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c4230007775-1095-bzdjz 6.12.55+ #1 SMP Sun Feb 1 08:59:41 UTC 2026 aarch64 GNU/Linux`
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
Comparing filter-pushdown-dynamic-bytes (290784a7a7630ae9bb0ba4f596110211024038a3) to ec00112 (merge-base) [diff](https://github.com/apache/datafusion/compare/ec001124829a928d7b5fe3df696592a2118dc44a..290784a7a7630ae9bb0ba4f596110211024038a3) using: clickbench_partitioned
Results will be posted here when complete
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4229959251
run benchmark clickbench_partitioned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-4230032165
π€ Benchmark completed (GKE) | [trigger](https://github.com/apache/datafusion/pull/20363#issuecomment-4230007606)
**Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
<details><summary>CPU Details (lscpu)</summary>
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Vendor ID: ARM
Model name: Neoverse-V2
Model: 1
Thread(s) per core: 1
Core(s) per cluster: 16
Socket(s): -
Cluster(s): 1
Stepping: r0p1
BogoMIPS: 2000.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache: 1 MiB (16 instances)
L1i cache: 1 MiB (16 instances)
L2 cache: 32 MiB (16 instances)
L3 cache: 80 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-15
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Mitigation; CSV2, BHB
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected
Vulnerability Vmscape: Not affected
```
</details>
<details><summary>Details</summary>
<p>
```
Comparing HEAD and filter-pushdown-dynamic-bytes
--------------------
Benchmark clickbench_partitioned.json
--------------------
βββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββββββββββββββββββββββββββ³ββββββββββββββββ
β Query β HEAD β filter-pushdown-dynamic-bytes β Change β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β QQuery 0 β 1.23 / 4.56 Β±6.47 / 17.49 ms β 1.19 / 4.45 Β±6.40 / 17.26 ms β no change β
β QQuery 1 β 14.40 / 15.04 Β±0.34 / 15.39 ms β 14.19 / 14.60 Β±0.22 / 14.82 ms β no change β
β QQuery 2 β 43.97 / 44.52 Β±0.38 / 45.02 ms β 43.95 / 44.04 Β±0.09 / 44.15 ms β no change β
β QQuery 3 β 46.53 / 47.96 Β±1.50 / 50.34 ms β 41.47 / 44.14 Β±2.71 / 48.34 ms β +1.09x faster β
β QQuery 4 β 322.09 / 356.01 Β±29.82 / 396.61 ms β 289.16 / 293.85 Β±3.57 / 298.65 ms β +1.21x faster β
β QQuery 5 β 367.65 / 375.06 Β±9.31 / 392.28 ms β 344.66 / 347.16 Β±2.71 / 350.68 ms β +1.08x faster β
β QQuery 6 β 5.22 / 6.56 Β±1.15 / 8.45 ms β 5.44 / 6.20 Β±1.03 / 8.19 ms β +1.06x faster β
β QQuery 7 β 17.60 / 18.39 Β±0.61 / 19.13 ms β 16.63 / 17.14 Β±0.81 / 18.76 ms β +1.07x faster β
β QQuery 8 β 431.53 / 448.69 Β±14.48 / 464.88 ms β 408.99 / 418.67 Β±8.56 / 428.94 ms β +1.07x faster β
β QQuery 9 β 689.73 / 705.60 Β±9.11 / 717.42 ms β 635.05 / 641.70 Β±6.77 / 652.58 ms β +1.10x faster β
β QQuery 10 β 95.27 / 98.34 Β±2.04 / 101.19 ms β 93.11 / 95.26 Β±2.37 / 99.71 ms β no change β
β QQuery 11 β 106.93 / 108.37 Β±1.53 / 110.93 ms β 103.34 / 104.43 Β±1.10 / 106.46 ms β no change β
β QQuery 12 β 363.01 / 368.83 Β±5.84 / 379.53 ms β 336.79 / 340.03 Β±2.10 / 342.42 ms β +1.08x faster β
β QQuery 13 β 473.83 / 487.77 Β±9.92 / 499.34 ms β 444.24 / 462.25 Β±10.24 / 472.48 ms β +1.06x faster β
β QQuery 14 β 355.55 / 364.06 Β±6.57 / 375.33 ms β 344.72 / 348.42 Β±4.08 / 353.45 ms β no change β
β QQuery 15 β 364.99 / 389.99 Β±25.67 / 434.73 ms β 342.38 / 356.76 Β±12.25 / 373.30 ms β +1.09x faster β
β QQuery 16 β 747.09 / 768.62 Β±22.29 / 809.71 ms β 702.78 / 715.35 Β±9.16 / 730.91 ms β +1.07x faster β
β QQuery 17 β 731.34 / 735.78 Β±2.60 / 738.79 ms β 707.50 / 713.09 Β±3.55 / 715.99 ms β no change β
β QQuery 18 β 1417.76 / 1482.77 Β±44.48 / 1545.36 ms β 1415.80 / 1460.83 Β±34.00 / 1509.06 ms β no change β
β QQuery 19 β 37.82 / 39.02 Β±0.86 / 40.35 ms β 35.20 / 57.30 Β±31.85 / 118.05 ms β 1.47x slower β
β QQuery 20 β 726.34 / 742.75 Β±16.82 / 773.05 ms β 724.53 / 734.75 Β±15.57 / 765.37 ms β no change β
β QQuery 21 β 771.72 / 777.72 Β±3.63 / 781.24 ms β 771.14 / 779.97 Β±10.95 / 800.19 ms β no change β
β QQuery 22 β 1139.59 / 1149.87 Β±8.20 / 1157.86 ms β 1128.68 / 1134.17 Β±4.36 / 1139.90 ms β no change β
β QQuery 23 β 3098.92 / 3120.63 Β±26.73 / 3173.25 ms β 3019.54 / 3066.35 Β±24.15 / 3086.38 ms β no change β
β QQuery 24 β 99.94 / 102.71 Β±2.85 / 107.74 ms β 98.58 / 101.72 Β±2.17 / 105.25 ms β no change β
β QQuery 25 β 139.90 / 140.99 Β±1.12 / 142.55 ms β 137.90 / 140.34 Β±1.91 / 143.27 ms β no change β
β QQuery 26 β 97.36 / 101.13 Β±2.03 / 103.30 ms β 98.10 / 99.85 Β±1.45 / 102.19 ms β no change β
β QQuery 27 β 863.48 / 866.12 Β±2.18 / 869.84 ms β 848.21 / 854.23 Β±6.34 / 863.02 ms β no change β
β QQuery 28 β 3286.82 / 3323.92 Β±26.10 / 3365.41 ms β 3258.63 / 3293.46 Β±20.85 / 3314.43 ms β no change β
β QQuery 29 β 51.46 / 56.66 Β±5.38 / 66.67 ms β 50.52 / 54.94 Β±3.07 / 59.93 ms β no change β
β QQuery 30 β 360.59 / 371.96 Β±7.25 / 382.36 ms β 353.89 / 363.82 Β±7.51 / 376.40 ms β no change β
β QQuery 31 β 362.89 / 381.43 Β±10.69 / 392.73 ms β 349.33 / 369.24 Β±10.85 / 379.49 ms β no change β
β QQuery 32 β 1050.52 / 1081.85 Β±34.99 / 1140.62 ms β 1208.62 / 1260.34 Β±43.86 / 1335.44 ms β 1.16x slower β
β QQuery 33 β 1452.30 / 1485.80 Β±18.08 / 1502.12 ms β 1475.63 / 1548.82 Β±51.96 / 1615.63 ms β no change β
β QQuery 34 β 1449.79 / 1486.00 Β±33.36 / 1528.07 ms β 1499.06 / 1533.16 Β±21.12 / 1562.02 ms β no change β
β QQuery 35 β 398.84 / 405.03 Β±5.09 / 414.14 ms β 395.02 / 426.78 Β±30.37 / 468.67 ms β 1.05x slower β
β QQuery 36 β 113.25 / 120.77 Β±5.45 / 127.28 ms β 119.28 / 122.66 Β±2.04 / 124.75 ms β no change β
β QQuery 37 β 48.13 / 49.92 Β±1.43 / 51.38 ms β 47.72 / 49.33 Β±1.39 / 51.36 ms β no change β
β QQuery 38 β 75.82 / 77.30 Β±1.46 / 79.54 ms β 75.46 / 77.53 Β±2.57 / 82.44 ms β no change β
β QQuery 39 β 215.66 / 223.02 Β±5.85 / 230.46 ms β 210.13 / 216.32 Β±3.44 / 219.91 ms β no change β
β QQuery 40 β 23.52 / 26.20 Β±1.68 / 27.87 ms β 23.68 / 25.71 Β±1.45 / 27.99 ms β no change β
β QQuery 41 β 19.92 / 21.54 Β±1.35 / 23.86 ms β 20.51 / 21.42 Β±0.96 / 23.27 ms β no change β
β QQuery 42 β 20.59 / 21.02 Β±0.32 / 21.56 ms β 19.43 / 19.98 Β±0.36 / 20.55 ms β no change β
βββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββ
ββββββββββββββββββββββββββββββββββββββββββββββββ³βββββββββββββ
β Benchmark Summary β β
β‘ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ©
β Total Time (HEAD) β 23000.26ms β
β Total Time (filter-pushdown-dynamic-bytes) β 22780.56ms β
β Average Time (HEAD) β 534.89ms β
β Average Time (filter-pushdown-dynamic-bytes) β 529.78ms β
β Queries Faster β 11 β
β Queries Slower β 3 β
β Queries with No Change β 29 β
β Queries with Failure β 0 β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββ
```
</p>
</details>
<details><summary>Resource Usage</summary>
**clickbench_partitioned β base (merge-base)**
| Metric | Value |
|--------|-------|
| Wall time | 116.1s |
| Peak memory | 40.3 GiB |
| Avg memory | 31.7 GiB |
| CPU user | 1099.6s |
| CPU sys | 86.3s |
| Peak spill | 0 B |
**clickbench_partitioned β branch**
| Metric | Value |
|--------|-------|
| Wall time | 114.6s |
| Peak memory | 38.0 GiB |
| Avg memory | 27.6 GiB |
| CPU user | 1070.0s |
| CPU sys | 96.9s |
| Peak spill | 0 B |
</details>
---
[File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977438987
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (d3711b9f88171d95055fb97a25317a2f4d6c159e) to acec058cb5e32c7a339280d7b58ebe58d8b38e8b [diff](https://github.com/apache/datafusion/compare/acec058cb5e32c7a339280d7b58ebe58d8b38e8b..d3711b9f88171d95055fb97a25317a2f4d6c159e) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977438776
run benchmark clickbench_partitioned
DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true
DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3977532475
π€ Hi @adriangb, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3977531640).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20613_3977496772.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20613#issuecomment-3977496772` |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979871835
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979874686
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3979875055
π€ Hi @Dandandan, you asked to view the benchmark queue (https://github.com/apache/datafusion/pull/20363#issuecomment-3979870119).
| Job | User | Benchmarks | Comment |
| --- | --- | --- | --- |
| `20363_3977503986.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977503986` |
| `20363_3977504315.sh` | adriangb | tpch (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20363#issuecomment-3977504315` |
| `20447_3977530468.sh` | adriangb | tpcds | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977530468` |
| `20447_3977531157.sh` | adriangb | tpcds (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20447#issuecomment-3977531157` |
| `20621_3977593319.sh` | Dandandan | default | `https://github.com/apache/datafusion/pull/20621#issuecomment-3977593319` |
| `arrow-9464-3977709353.sh` | Dandandan | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/9464#issuecomment-3977709353` |
| `20481_3978024622.sh` | Dandandan | clickbench_partitioned (env: DATAFUSION_EXECUTION_PARQUET_PUSHDOWN_FILTERS=true DATAFUSION_EXECUTION_PARQUET_REORDER_FILTERS=true) | `https://github.com/apache/datafusion/pull/20481#issuecomment-3978024622` |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960455042
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (aaa75f256caa1f3c66d9c91ae563f2b2635c563d) to b9328b9734534c8e2ce92b00c5368f88956bf5c3 [diff](https://github.com/apache/datafusion/compare/b9328b9734534c8e2ce92b00c5368f88956bf5c3..aaa75f256caa1f3c66d9c91ae563f2b2635c563d) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3960831558
π€ `./gh_compare_branch.sh` [gh_compare_branch.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_branch.sh) Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing filter-pushdown-dynamic-bytes (3a4511fb565ca558d6101cda7a293613e0d743c5) to b9328b9734534c8e2ce92b00c5368f88956bf5c3 [diff](https://github.com/apache/datafusion/compare/b9328b9734534c8e2ce92b00c5368f88956bf5c3..3a4511fb565ca558d6101cda7a293613e0d743c5) using: clickbench_partitioned
Results will be posted here when complete
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] (Test) Advanced adaptive filter selectivity evaluation [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20363:
URL: https://github.com/apache/datafusion/pull/20363#issuecomment-3947177189
show benchmark queue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org