[GitHub] [arrow-rs] kazuk commented on issue #1274: Automatically run and track arrow / parquet benchmarks overtime

[GitBox <gi...@apache.org>]

kazuk commented on issue #1274:
URL: https://github.com/apache/arrow-rs/issues/1274#issuecomment-1124425450

   I write bash script for this.
   
   usage example `./dev/bench/run_bench.sh 11.0.0 12.0.0`
   
   This saves 4 baseline: arrow-11.0.0 and arrow-12.0.0 , parquet-11.0.0, parquet-12.0.0,
   
   this can compare with [critcmp](https://github.com/BurntSushi/critcmp)
   
   ```bash
   #!/bin/bash
   #
   # run bench for 2 git hash/branch/tag
   #
   # ./dev/bench/run_bench.sh [baseline] [compare]
   
   set -e
   
   function patch_cargo_toml() {
       if [ ! -z $(grep "bench = false" arrow/Cargo.toml) ]; then 
           echo "Patch not required"
       else
           echo "Patch required"
           cp arrow/Cargo.toml arrow/Cargo.toml.org
           sed '/^\[lib\]/a bench = false' arrow/Cargo.toml.org > arrow/Cargo.toml
           echo '[lib]' >> parquet/Cargo.toml
           echo 'bench = false' >> parquet/Cargo.toml
       fi
   }
   
   function run_bench( ) {
       cd arrow
       cargo bench -- --save-baseline arrow-$1
       cd ..
       cd parquet
       cargo bench -- --save-baseline parquet-$1
       cd ..
   }
   
   
   # checkout baseline and run bench 
   git checkout $1
   
   patch_cargo_toml 
   run_bench $1
   
   # checkout compare and run bench
   git reset --hard
   git checkout $2
   
   patch_cargo_toml 
   run_bench $2
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org