You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@cassandra.apache.org by Andrés de la Peña <ad...@apache.org> on 2021/05/12 11:45:58 UTC

CircleCI job to run some tests repeatedly

Hi all,

Just to inform that CASSANDRA-16625 has been merged. It adds new CircleCI
jobs to run a specific test repeatedly, so we can check if it's stable.
Hopefully it can be helpful to reproduce failures when fixing a flaky test,
and to verify that a new or modified test is stable.

JUnit tests, such as unit tests or JVM dtests, can be run with the
"repeated_utest" job, while Python dtests can be run with the
"repeated_dtest" job. The names of the tests to be run, the number of
repetitions, etc. are set in env vars in the CircleCI config file. There
are more details about these vars in the documentation:
https://github.com/apache/cassandra/blob/trunk/doc/source/development/testing.rst#circleci

For example, this job runs the StreamingMetricsTest JVM dtest 200 times in
~5min, hitting 23 failures:
https://app.circleci.com/pipelines/github/adelapena/cassandra/388/workflows/034b59a0-11ed-469f-bb24-d13922f8a942/jobs/3346

Regards,

Re: CircleCI job to run some tests repeatedly

Posted by Ekaterina Dimitrova <e....@gmail.com>.

Thanks Andres, I am already using it!
Best regards,
Ekaterina

On Wed, 12 May 2021 at 8:02, Benjamin Lerer <be...@datastax.com>
wrote:

> And you had to pick a test I wrote as an example of a flaky one ;-)
>
> Thanks Andrés. Those new jobs will be super useful.
>
> On Wed, May 12, 2021 at 1:46 PM Andrés de la Peña <ad...@apache.org>
> wrote:
>
> > Hi all,
> >
> > Just to inform that CASSANDRA-16625 has been merged. It adds new CircleCI
> > jobs to run a specific test repeatedly, so we can check if it's stable.
> > Hopefully it can be helpful to reproduce failures when fixing a flaky
> test,
> > and to verify that a new or modified test is stable.
> >
> > JUnit tests, such as unit tests or JVM dtests, can be run with the
> > "repeated_utest" job, while Python dtests can be run with the
> > "repeated_dtest" job. The names of the tests to be run, the number of
> > repetitions, etc. are set in env vars in the CircleCI config file. There
> > are more details about these vars in the documentation:
> >
> >
> https://github.com/apache/cassandra/blob/trunk/doc/source/development/testing.rst#circleci
> >
> > For example, this job runs the StreamingMetricsTest JVM dtest 200 times
> in
> > ~5min, hitting 23 failures:
> >
> >
> https://app.circleci.com/pipelines/github/adelapena/cassandra/388/workflows/034b59a0-11ed-469f-bb24-d13922f8a942/jobs/3346
> >
> > Regards,
> >
>

Re: CircleCI job to run some tests repeatedly

Posted by Benjamin Lerer <be...@datastax.com>.

And you had to pick a test I wrote as an example of a flaky one ;-)

Thanks Andrés. Those new jobs will be super useful.

On Wed, May 12, 2021 at 1:46 PM Andrés de la Peña <ad...@apache.org>
wrote:

> Hi all,
>
> Just to inform that CASSANDRA-16625 has been merged. It adds new CircleCI
> jobs to run a specific test repeatedly, so we can check if it's stable.
> Hopefully it can be helpful to reproduce failures when fixing a flaky test,
> and to verify that a new or modified test is stable.
>
> JUnit tests, such as unit tests or JVM dtests, can be run with the
> "repeated_utest" job, while Python dtests can be run with the
> "repeated_dtest" job. The names of the tests to be run, the number of
> repetitions, etc. are set in env vars in the CircleCI config file. There
> are more details about these vars in the documentation:
>
> https://github.com/apache/cassandra/blob/trunk/doc/source/development/testing.rst#circleci
>
> For example, this job runs the StreamingMetricsTest JVM dtest 200 times in
> ~5min, hitting 23 failures:
>
> https://app.circleci.com/pipelines/github/adelapena/cassandra/388/workflows/034b59a0-11ed-469f-bb24-d13922f8a942/jobs/3346
>
> Regards,
>