You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Valentyn Tymofieiev (Jira)" <ji...@apache.org> on 2019/09/10 20:45:00 UTC

[jira] [Created] (BEAM-8199) Improve tooling to enable alerts on performance regression based on data collected from Beam benchmarks

Valentyn Tymofieiev created BEAM-8199:
-----------------------------------------

             Summary: Improve tooling to enable alerts on performance regression based on data collected from Beam benchmarks
                 Key: BEAM-8199
                 URL: https://issues.apache.org/jira/browse/BEAM-8199
             Project: Beam
          Issue Type: Bug
          Components: testing
            Reporter: Valentyn Tymofieiev


Currently we have some performance benchmarks in Beam, however they are not always monitored and we may  not notice when a regression happens, see [1]. 

Filing this issue to improve the tooling. Note that if automation becomes too noisy (too many regressions) people likely won't be paying attention either.

As a first step, we could monitor regression in 50th percentile of performance metrics we collect for benchmarks that we run repeatedly.

cc:  [~kamilwu] [~kasiak] [~markflyhigh]

https://lists.apache.org/thread.html/51e000f16481451c207c00ac5e881aa4a46fa020922eddffd00ad527@%3Cdev.beam.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.2#803003)