You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andres de la Peña (Jira)" <ji...@apache.org> on 2022/10/05 10:59:00 UTC
[jira] [Comment Edited] (CASSANDRA-17939) CircleCI: Automatically detect and repeat new or modified JUnit tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612943#comment-17612943 ]
Andres de la Peña edited comment on CASSANDRA-17939 at 10/5/22 10:58 AM:
-------------------------------------------------------------------------
[The proposed PR|https://github.com/apache/cassandra/pull/1899] adds new jobs for automatically repeating new or modified JUnit tests. Each regular JUnit job has a companion job to run the tests on that suite that are new or modified, according to the results of {{{}git diff --diff-filter=AMR{}}}. Each pair of all/repeated jobs is subjected to the same approval step, so the new workflow has more jobs but an identical number of approval steps.
These CI results use a synthetic patch doing some arbitrary changes on every test family, so we can verify how the new jobs work:
||PR||Test patch||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-dev-test]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/e54ff736-015e-4002-b170-6ea826b591d1]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/c2811225-9a38-4216-9b7e-b15c9bb6d53c]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/13593914-4cdf-4f88-b5be-3201908e3e01]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/7c7d779d-0d22-4977-abca-0ddf01adb327]|
I'll add PRs for the remaining branches once we settle on a design.
This ticket is focused only on JUnit tests. Similar jobs for Python tests can be added in a separate ticket if we see value on it. However, that would be quite more challenging because we would need to find a way to get the class and method name of the new or modified test files identified by {{{}git diff{}}}. We probably shouldn't run entire {{.py}} files in the same way that we run {{.java}} files because Python tests are more resource intensive and their test files use to be quite crowded. To further complicate things, the names of some Python tests are automatically generated and don't appear on the compared files. Also, adding Python tests is not as common as adding JUnit tests these days, so there is probably less value on automatically multiplexing them.
was (Author: adelapena):
[The proposed PR|https://github.com/apache/cassandra/pull/1899] adds new jobs for automatically repeating new or modified JUnit tests. Each regular JUnit job has a companion job to run the tests on that suite that are new or modified, according to the results of {{{}git diff --diff-filter=AMR{}}}. Each pair of all/repeated jobs is subjected to the same approval step, so the new workflow has more jobs but an identical number of approval steps.
These CI results use a synthetic patch doing some arbitrary changes on every test family, so we can verify how the new jobs work:
||PR||Test patch||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-dev-test]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/e54ff736-015e-4002-b170-6ea826b591d1]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/c2811225-9a38-4216-9b7e-b15c9bb6d53c]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/13593914-4cdf-4f88-b5be-3201908e3e01]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/7c7d779d-0d22-4977-abca-0ddf01adb327]|
This ticket is focused only on JUnit tests. Similar jobs for Python tests can be added in a separate ticket if we see value on it. However, that would be quite more challenging because we would need to find a way to get the class and method name of the new or modified test files identified by {{{}git diff{}}}. We probably shouldn't run entire {{.py}} files in the same way that we run {{.java}} files because Python tests are more resource intensive and their test files use to be quite crowded. To further complicate things, the names of some Python tests are automatically generated and don't appear on the compared files. Also, adding Python tests is not as common as adding JUnit tests these days, so there is probably less value on automatically multiplexing them.
> CircleCI: Automatically detect and repeat new or modified JUnit tests
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-17939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17939
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Andres de la Peña
> Priority: Normal
>
> The purpose of this ticket is adding a new CircleCI job that automatically detects new or modified test classes and runs them repeatedly. That way we wouldn't need to manually specify those tests with {{.circleci/generate.sh}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org