[jira] [Assigned] (FLINK-27571) Recognize "less is better" benchmarks in regression detection script

["Roman Khachatryan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-27571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Roman Khachatryan reassigned FLINK-27571:
-----------------------------------------

    Assignee: Roman Khachatryan

> Recognize "less is better" benchmarks in regression detection script
> --------------------------------------------------------------------
>
>                 Key: FLINK-27571
>                 URL: https://issues.apache.org/jira/browse/FLINK-27571
>             Project: Flink
>          Issue Type: Bug
>          Components: Benchmarks
>    Affects Versions: 1.16.0
>            Reporter: Roman Khachatryan
>            Assignee: Roman Khachatryan
>            Priority: Major
>         Attachments: Screenshot_2022-05-09_10-33-11.png
>
>
> Example benchmark:
> [http://codespeed.dak8s.net:8000/timeline/#/?exe=5&ben=schedulingDownstreamTasks.BATCH&extr=on&quarts=on&equid=off&env=2&revs=200]
>  
> [Proposed solution|https://issues.apache.org/jira/browse/FLINK-27555?focusedCommentId=17534423&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17534423]:
> {quote}
> I think #2 is the correct way.
> Maybe we can modify the save_jmh_result.py to correctly set the 'units' and the 'lessisbetter' fields of benchmark results. The 'units' is already contained in the jmh result and the 'lessisbetter' can be derived from the mode(false if it is 'thrpt' mode, otherwise true). An example of the jmh result format can be found at https://i.stack.imgur.com/vB3fV.png.
> This can fix the web UI as well as the REST result, and then the regression_report.py will be able to identify which benchmarks are "less is better" and treat them differently.
> {quote}
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)