[GitHub] [iceberg] mr-brobot commented on pull request #8104: Python: Optimize concurrency for limited queries

["mr-brobot (via GitHub)" <gi...@apache.org>]

mr-brobot commented on PR #8104:
URL: https://github.com/apache/iceberg/pull/8104#issuecomment-1679079195

   Latest benchmark results with same table (~1.5B record table in S3 with ~230 data files):
   
   On `master`
   ```
   $ hyperfine --warmup 1 "python -m scripts.pyiceberg"
   Benchmark 1: python -m scripts.pyiceberg
     Time (mean ± σ):      8.661 s ±  0.313 s    [User: 17.781 s, System: 6.045 s]
     Range (min … max):    8.355 s …  9.256 s    10 runs
   ```
   
   On this branch
   ```
   $ hyperfine --warmup 1 "python -m scripts.pyiceberg"
   Benchmark 1: python -m scripts.pyiceberg
     Time (mean ± σ):      8.517 s ±  0.260 s    [User: 17.737 s, System: 6.429 s]
     Range (min … max):    8.224 s …  8.917 s    10 runs
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org