You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/04/30 22:57:56 UTC
[I] tracking tpc-h sf=100 benchmark results [datafusion-ballista]
andygrove opened a new issue, #1632:
URL: https://github.com/apache/datafusion-ballista/issues/1632
I'm running TPC-H benchmarks and wanted somewhere to keep track of the the results. No action is needed for this issue.
Setup:
```shell
RUST_LOG=info ./target/release/ballista-scheduler
RUST_LOG=info ./target/release/ballista-executor -c 16
```
Benchmark command:
```shell
./target/release/tpch benchmark ballista --host localhost --port 50050 --path /mnt/bigdata/tpch/sf100 --format parquet --iterations 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] tracking tpc-h sf=100 benchmark results [datafusion-ballista]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1632:
URL: https://github.com/apache/datafusion-ballista/issues/1632#issuecomment-4356806508
With sort-based shuffle disabled, it is much faster, confusingly.
```
Query 1 took 19427.7 ms and returned 4 rows
Query 2 took 6971.9 ms and returned 100 rows
Query 3 took 12378.4 ms and returned 10 rows
Query 4 took 6086.6 ms and returned 5 rows
Query 5 took 29025.9 ms and returned 5 rows
Query 6 took 4478.6 ms and returned 1 rows
Query 7 took 44554.4 ms and returned 4 rows
Query 8 took 29580.4 ms and returned 2 rows
Query 9 took 49119.9 ms and returned 175 rows
Query 10 took 15762.9 ms and returned 20 rows
Query 11 took 4772.3 ms and returned 0 rows
Query 12 took 10083.2 ms and returned 2 rows
Query 13 took 17134.2 ms and returned 45 rows
Query 14 took 4923.0 ms and returned 1 rows
Query 15 took 5867.2 ms and returned 0 rows
Query 16 took 3431.7 ms and returned 27840 rows
Query 17 took 45181.6 ms and returned 1 rows
Query 18 took 52234.5 ms and returned 100 rows
Query 19 took 12596.4 ms and returned 1 rows
Query 20 took 10437.2 ms and returned 17971 rows
Query 21 took 51066.2 ms and returned 100 rows
Query 22 took 3160.8 ms and returned 7 rows
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] tracking tpc-h sf=100 benchmark results [datafusion-ballista]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #1632: tracking tpc-h sf=100 benchmark results
URL: https://github.com/apache/datafusion-ballista/issues/1632
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] tracking tpc-h sf=100 benchmark results [datafusion-ballista]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1632:
URL: https://github.com/apache/datafusion-ballista/issues/1632#issuecomment-4357709068
PR https://github.com/apache/datafusion-ballista/pull/1636 fixes the major issue that was causing terrible results with sort-based shuffle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] tracking tpc-h sf=100 benchmark results [datafusion-ballista]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1632:
URL: https://github.com/apache/datafusion-ballista/issues/1632#issuecomment-4356736760
After fixing the recent regression in sort-based shuffle, running with default configs:
```
Query 1 took 19147.2 ms and returned 4 rows
Query 2 took 10234.8 ms and returned 100 rows
Query 3 took 73433.7 ms and returned 10 rows
Query 4 took 15675.1 ms and returned 5 rows
Query 5 took 190789.9 ms and returned 5 rows
Query 6 took 4440.7 ms and returned 1 rows
Query 7 took 77920.9 ms and returned 4 rows
Query 8 took 262859.9 ms and returned 2 rows
Query 9 took 341142.4 ms and returned 175 rows
Query 10 took 20336.1 ms and returned 20 rows
Query 11 took 8026.6 ms and returned 0 rows
Query 12 took 10911.1 ms and returned 2 rows
Query 13 took 18763.3 ms and returned 45 rows
Query 14 took 5046.3 ms and returned 1 rows
Query 15 took 6356.9 ms and returned 0 rows
Query 16 took 4254.7 ms and returned 27840 rows
Query 17 took 205500.7 ms and returned 1 rows
Query 18 took 106379.5 ms and returned 100 rows
Query 19 took 13369.0 ms and returned 1 rows
Query 20 took 12163.2 ms and returned 17971 rows
Query 21 took 188630.3 ms and returned 100 rows
Query 22 took 4723.8 ms and returned 7 rows
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org