You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/04/26 21:10:28 UTC
[GitHub] [arrow-datafusion] alamb commented on pull request #6131: Add bench.sh script to benchmark DataFusion against itself
alamb commented on PR #6131:
URL: https://github.com/apache/arrow-datafusion/pull/6131#issuecomment-1524050533
Some interesting results already -- I ran a quick experiment to see how much 'lto' link time optimization helps. The answer is "quite a bit"
```
alamb@aal-dev:~/arrow-datafusion2/benchmarks$ python compare.py results/alamb_bench/tpch.json results/alamb_bench_compare/tpch.json
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ /home/alamb… ┃ /home/alamb… ┃ Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 1269.30ms │ 1097.60ms │ +1.16x faster │
│ QQuery 2 │ 418.17ms │ 309.01ms │ +1.35x faster │
│ QQuery 3 │ 393.15ms │ 365.30ms │ +1.08x faster │
│ QQuery 4 │ 212.83ms │ 214.36ms │ no change │
│ QQuery 5 │ 534.56ms │ 531.17ms │ no change │
│ QQuery 6 │ 209.39ms │ 184.41ms │ +1.14x faster │
│ QQuery 7 │ 1037.82ms │ 981.81ms │ +1.06x faster │
│ QQuery 8 │ 550.99ms │ 540.95ms │ no change │
│ QQuery 9 │ 982.00ms │ 984.53ms │ no change │
│ QQuery 10 │ 613.48ms │ 560.14ms │ +1.10x faster │
│ QQuery 11 │ 272.45ms │ 231.46ms │ +1.18x faster │
│ QQuery 12 │ 319.91ms │ 320.54ms │ no change │
│ QQuery 13 │ 1127.46ms │ 1087.70ms │ no change │
│ QQuery 14 │ 286.89ms │ 263.16ms │ +1.09x faster │
│ QQuery 15 │ 255.63ms │ 233.42ms │ +1.10x faster │
│ QQuery 16 │ 302.94ms │ 309.25ms │ no change │
│ QQuery 17 │ 2891.05ms │ 2628.59ms │ +1.10x faster │
│ QQuery 18 │ 3123.23ms │ 3154.47ms │ no change │
│ QQuery 19 │ 511.97ms │ 472.75ms │ +1.08x faster │
│ QQuery 20 │ 1042.76ms │ 938.75ms │ +1.11x faster │
│ QQuery 21 │ 1567.78ms │ 1611.91ms │ no change │
│ QQuery 22 │ 182.56ms │ 171.52ms │ +1.06x faster │
└──────────────┴──────────────┴──────────────┴───────────────┘
alamb@aal-dev:~/arrow-datafusion2/benchmarks$ python compare.py results/alamb_bench/tpch_mem.json results/alamb_bench_compare/tpch_mem.json
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ -o ┃ -o ┃ Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1 │ 876.15ms │ 796.57ms │ +1.10x faster │
│ QQuery 2 │ 265.87ms │ 267.19ms │ no change │
│ QQuery 3 │ 169.43ms │ 164.82ms │ no change │
│ QQuery 4 │ 110.07ms │ 116.02ms │ 1.05x slower │
│ QQuery 5 │ 462.34ms │ 449.41ms │ no change │
│ QQuery 6 │ 44.40ms │ 40.54ms │ +1.10x faster │
│ QQuery 7 │ 1099.47ms │ 1077.84ms │ no change │
│ QQuery 8 │ 241.97ms │ 247.20ms │ no change │
│ QQuery 9 │ 584.01ms │ 606.74ms │ no change │
│ QQuery 10 │ 301.95ms │ 299.14ms │ no change │
│ QQuery 11 │ 239.17ms │ 221.07ms │ +1.08x faster │
│ QQuery 12 │ 153.73ms │ 139.95ms │ +1.10x faster │
│ QQuery 13 │ 793.76ms │ 753.25ms │ +1.05x faster │
│ QQuery 14 │ 59.50ms │ 49.38ms │ +1.20x faster │
│ QQuery 15 │ 103.03ms │ 89.82ms │ +1.15x faster │
│ QQuery 16 │ 216.38ms │ 213.85ms │ no change │
│ QQuery 17 │ 3356.99ms │ 2866.85ms │ +1.17x faster │
│ QQuery 18 │ 3017.82ms │ 2910.50ms │ no change │
│ QQuery 19 │ 161.19ms │ 137.81ms │ +1.17x faster │
│ QQuery 20 │ 924.38ms │ 855.74ms │ +1.08x faster │
│ QQuery 21 │ 1502.52ms │ 1460.54ms │ no change │
│ QQuery 22 │ 133.10ms │ 128.34ms │ no change │
└──────────────┴──────────────┴──────────────┴───────────────┘
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org