[jira] [Created] (CASSANDRA-16006) Parallelise Jenkins dtests

["Michael Semb Wever (Jira)" <ji...@apache.org>]

Michael Semb Wever created CASSANDRA-16006:
----------------------------------------------

             Summary: Parallelise Jenkins dtests
                 Key: CASSANDRA-16006
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
             Project: Cassandra
          Issue Type: Improvement
          Components: CI
            Reporter: Michael Semb Wever


Currently dtests in Jenkins take ~10 hours.

Using the Jenkins Matrix plugin these jobs can be split into smaller lists of dtests and run in parallel. This is the approach CircleCI takes.

This approach was [trialed|https://github.com/apache/cassandra-builds/pull/29] with the dtest-upgrade jobs (which are not yet part of the branch pipelines, and haven't previously worked at all due to their duration).

In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded plugins also needed to be added.

The splits will occupy all executors, and multiple builds will lead to a long build queue. More important builds (artifacts and unit tests) need a way to be prioritised in such saturated situations.

Splits can fail for silly reasons (false-positive), like full /tmp disks, or connectivity issues between the donated agent servers. The Matrix Reloaded plugin makes it easy to rebuilt just those failed splits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org