You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Michael Semb Wever (Jira)" <ji...@apache.org> on 2020/08/03 07:56:00 UTC
[jira] [Comment Edited] (CASSANDRA-16006) Parallelise Jenkins
dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169608#comment-17169608 ]
Michael Semb Wever edited comment on CASSANDRA-16006 at 8/3/20, 7:55 AM:
-------------------------------------------------------------------------
[Patch|https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-dtest-parallel] ready. Tested locally.
When committing this, all existing dtest (mainline branch dtest-upgrade excluded) jobs will first need to be renamed, as the job type is changing.
was (Author: michaelsembwever):
Patch ready. Tested locally.
When committing this, all existing dtest (mainline branch dtest-upgrade excluded) jobs will first need to be renamed, as the job type is changing.
> Parallelise Jenkins dtests
> --------------------------
>
> Key: CASSANDRA-16006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
> Project: Cassandra
> Issue Type: Improvement
> Components: CI
> Reporter: Michael Semb Wever
> Assignee: Michael Semb Wever
> Priority: Normal
> Fix For: 4.0-rc
>
>
> Currently dtests in Jenkins take ~10 hours.
> Using the Jenkins Matrix plugin these jobs can be split into smaller lists of dtests and run in parallel. This is the approach CircleCI takes.
> This approach was [trialed|https://github.com/apache/cassandra-builds/pull/29] with the dtest-upgrade jobs (which are not yet part of the branch pipelines, and haven't previously worked at all due to their duration).
> In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded plugins also needed to be added.
> The splits will occupy all executors, and multiple builds will lead to a long build queue. More important builds (artifacts and unit tests) need a way to be prioritised in such saturated situations.
> Splits can fail for silly reasons (false-positive), like [full|https://issues.apache.org/jira/browse/INFRA-19793] /tmp disks, or [connectivity issues|https://issues.apache.org/jira/browse/INFRA-20107] between the donated agent servers. The Matrix Reloaded plugin makes it easy to rebuilt just those failed splits.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org