[GitHub] WireBaron opened a new pull request #7: GEODE-6086: Adding a tool to allow command line analysis of test runs

[GitBox <gi...@apache.org>]

WireBaron opened a new pull request #7: GEODE-6086: Adding a tool to allow command line analysis of test runs
URL: https://github.com/apache/geode-benchmarks/pull/7
 
 
   This change adds the analyzeRun gradle target, which, when passed a baseline
   result directory and test result directory via --args, will analyze the output
   and report the variation of the test from the baseline.
   
   
   Sample output: 
   ```
   MacBook-Pro-2:geode-performance browe$ ./gradlew analyzeRun --args "/Users/browe/project/geode-performance/geode-benchmarks/output2 /Users/browe/project/geode-performance/geode-benchmarks/output"
   
   > Task :harness:analyzeRun
   Running analyzer
   Comparing test result at /Users/browe/project/geode-performance/geode-benchmarks/output2 to baseline at /Users/browe/project/geode-performance/geode-benchmarks/output
   -- PartitionedPutBenchmark --
   average ops/second
   Result: 47386.93220338983
   Baseline: 46237.745762711864
   Relative performance: 1.0248538595842343
   
   99th percentile latency
   Result: 292.85714285714283
   Baseline: 294.11764705882354
   Relative performance: 0.9957142857142857
   ```
   
   Two remaining pain points I'd like to address:
   1. When adding a benchmark it would be easier to use roles rather than node names
   2. It would also be nice to reference the benchmark classes directly in the analyzer (to get their output directories), but that dependency goes the wrong way.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services