[jira] [Updated] (BEAM-12672) Update all Precommits to rerun failed tests K times.

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-12672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-12672:
-----------------------------------

This Jira ticket has a pull request attached to it, but is still open. Did the pull request resolve the issue? If so, could you please mark it resolved? This will help the project have a clear view of its open issues.

> Update all Precommits to rerun failed tests K times.
> ----------------------------------------------------
>
>                 Key: BEAM-12672
>                 URL: https://issues.apache.org/jira/browse/BEAM-12672
>             Project: Beam
>          Issue Type: Test
>          Components: testing
>            Reporter: Alex Amato
>            Assignee: Benjamin Gonzalez
>            Priority: P2
>          Time Spent: 11h 40m
>  Remaining Estimate: 0h
>
> We should configure the precommit test suites to re-run failed tests a few (K) times.
> (A bonus feature would be to tag certain tests as known flakey, and run multiple attempts in parallel when the test is first ran)
> The benefits of this are to:
>  * Not have flakey tests slow down PR reviews and merges.
>  * And avoiding the need to repeatedly manually rerun the precommit test with a PR comment.
>  * Getting a clear signal that failed tests are due to the code in the PR
>  * Save computing resources (Rather than needing to run the entire test suite every time it fails, only the failed tests will rerun).
> (Its not realistic to have 0 flakeyness, so this is desirable to improve productivity, as it can take several attempts to have all the tests pass in a single run).
> Note: This does not mean that we should ignore flakey tests, which are sometimes indicative of bugs. We should continue to investigate those with a continuous test suite.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)