You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andres de la Peña (Jira)" <ji...@apache.org> on 2021/05/12 16:23:00 UTC
[jira] [Comment Edited] (CASSANDRA-16654) Junit RepeatableRunner
flaky tests helper
[ https://issues.apache.org/jira/browse/CASSANDRA-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343343#comment-17343343 ]
Andres de la Peña edited comment on CASSANDRA-16654 at 5/12/21, 4:22 PM:
-------------------------------------------------------------------------
I think this is a nice addition to our tool belt, thanks. It will be useful at least until we have JUnit 5's [@RepeatedTest|https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/RepeatedTest.html] available, someday.
Maybe we could add a few implementations for our most common runners, as proposed [here|https://github.com/adelapena/cassandra/commit/65cd829bc73e1a9232e11c0bcfbeaccee2536c53], so we can use annotations such as {{@RunWith(RepeatableRunner.Parameterized.class)}}, {{@RunWith(RepeatableRunner.BMUnitRunner.class)}}, etc., wdyt?
I would add a note in the documentation about how many tests alter singletons and so they don't work with the repeatable runner.
We could put the number of repetitions in a {{RepeatableRunner}} constant for better visibility, and maybe also add the possibility of reading the value from a JVM property, as it's done [here|https://github.com/adelapena/cassandra/blob/65cd829bc73e1a9232e11c0bcfbeaccee2536c53/test/unit/org/apache/cassandra/RepeatableRunner.java#L35]. The later is probably not so useful since we are nevertheless forced to temporary modify the code when setting the runner.
was (Author: adelapena):
I think this is a nice addition to our tool belt, thanks. It will be useful at least until we have JUnit 5's [@RepeatedTest|https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/RepeatedTest.html] available, someday.
Maybe we could add a few implementations for our most common runners, as proposed [here|https://github.com/adelapena/cassandra/commit/65cd829bc73e1a9232e11c0bcfbeaccee2536c53], so we can use annotations such as {{@RunWith(RepeatableRunner.Parameterized.class)}}, {{@RunWith(RepeatableRunner.BMUnitRunner.class)}}, etc., wdyt?
I would add a note in the documentation about how many tests alter singletons and so they don't work with the repeatable runner.
We could put the number of repetitions in a {{RepeatableRunner}} constant for better visibility, and maybe also add the possibility of reading the value from a JVM property, as it's done [here|https://github.com/adelapena/cassandra/blob/65cd829bc73e1a9232e11c0bcfbeaccee2536c53/test/unit/org/apache/cassandra/RepeatableRunner.java#L35]. The later is probably not so useful since we are force to temporary modify the code when setting the runner.
> Junit RepeatableRunner flaky tests helper
> -----------------------------------------
>
> Key: CASSANDRA-16654
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16654
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
> Reporter: Berenguer Blasi
> Assignee: Berenguer Blasi
> Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.1
>
>
> Some flakies only fail when the full suite is ran. If it's a quick test you can do thousands of runs if a few seconds. Looping at the sh level is not an option as every loop takes a few seconds.
> As I have found this very useful I am proposing committing it
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org