[jira] [Updated] (ARROW-9239) [C++][Developer] Implement tool to compile and run C++ benchmarks from master branch against older codebase revisions

["Wes McKinney (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney updated ARROW-9239:
--------------------------------
    Summary: [C++][Developer] Implement tool to compile and run C++ benchmarks from master branch against older codebase revisions  (was: [C++][Developer] Implement tool to compile benchmark C++ code from master branch against older codebase revisions)

> [C++][Developer] Implement tool to compile and run C++ benchmarks from master branch against older codebase revisions
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: ARROW-9239
>                 URL: https://issues.apache.org/jira/browse/ARROW-9239
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++, Developer Tools
>            Reporter: Wes McKinney
>            Priority: Major
>
> Whenever a benchmark configuration is changed, the results from that benchmark become non-comparable with prior revisions of the benchmark before the change. This impacts "archery benchmark diff" and any other workflow that prepares benchmark comparisons
> I suggest the creation of a tool that 
> * Backports -benchmark.cc changes along with known dependent files (like stuff in arrow/testing) to a branch based on a target commit to benchmark
> * Compile as many of the benchmarks as possible (some may not compile due to API changes)
> * Run benchmarks and collect their data
> I've already done this manually in a number of cases to compare performance in PRs where there are changes to the -benchmark.cc files, but it would be better to have a more structured tool that simplifies this process



--
This message was sent by Atlassian Jira
(v8.3.4#803005)