You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andres de la Peña (Jira)" <ji...@apache.org> on 2021/05/12 16:23:00 UTC

[jira] [Comment Edited] (CASSANDRA-16654) Junit RepeatableRunner flaky tests helper

    [ https://issues.apache.org/jira/browse/CASSANDRA-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343343#comment-17343343 ] 

Andres de la Peña edited comment on CASSANDRA-16654 at 5/12/21, 4:22 PM:
-------------------------------------------------------------------------

I think this is a nice addition to our tool belt, thanks. It will be useful at least until we have JUnit 5's [@RepeatedTest|https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/RepeatedTest.html] available, someday.

Maybe we could add a few implementations for our most common runners, as proposed [here|https://github.com/adelapena/cassandra/commit/65cd829bc73e1a9232e11c0bcfbeaccee2536c53], so we can use annotations such as {{@RunWith(RepeatableRunner.Parameterized.class)}}, {{@RunWith(RepeatableRunner.BMUnitRunner.class)}}, etc., wdyt?

I would add a note in the documentation about how many tests alter singletons and so they don't work with the repeatable runner.

We could put the number of repetitions in a {{RepeatableRunner}} constant for better visibility, and maybe also add the possibility of reading the value from a JVM property, as it's done [here|https://github.com/adelapena/cassandra/blob/65cd829bc73e1a9232e11c0bcfbeaccee2536c53/test/unit/org/apache/cassandra/RepeatableRunner.java#L35]. The later is probably not so useful since we are nevertheless forced to temporary modify the code when setting the runner.


was (Author: adelapena):
I think this is a nice addition to our tool belt, thanks. It will be useful at least until we have JUnit 5's [@RepeatedTest|https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/RepeatedTest.html] available, someday.

Maybe we could add a few implementations for our most common runners, as proposed [here|https://github.com/adelapena/cassandra/commit/65cd829bc73e1a9232e11c0bcfbeaccee2536c53], so we can use annotations such as {{@RunWith(RepeatableRunner.Parameterized.class)}}, {{@RunWith(RepeatableRunner.BMUnitRunner.class)}}, etc., wdyt?

I would add a note in the documentation about how many tests alter singletons and so they don't work with the repeatable runner.

We could put the number of repetitions in a {{RepeatableRunner}} constant for better visibility, and maybe also add the possibility of reading the value from a JVM property, as it's done [here|https://github.com/adelapena/cassandra/blob/65cd829bc73e1a9232e11c0bcfbeaccee2536c53/test/unit/org/apache/cassandra/RepeatableRunner.java#L35]. The later is probably not so useful since we are force to temporary modify the code when setting the runner.

> Junit RepeatableRunner flaky tests helper
> -----------------------------------------
>
>                 Key: CASSANDRA-16654
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-16654
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Test/unit
>            Reporter: Berenguer Blasi
>            Assignee: Berenguer Blasi
>            Priority: Normal
>             Fix For: 4.0-rc2, 4.0, 4.1
>
>
> Some flakies only fail when the full suite is ran. If it's a quick test you can do thousands of runs if a few seconds. Looping at the sh level is not an option as every loop takes a few seconds.
> As I have found this very useful I am proposing committing it



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org