You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by ad...@apache.org on 2022/10/18 15:59:16 UTC
[cassandra] branch cassandra-3.0 updated: CircleCI: Automatically detect and repeat new or modified JUnit tests
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 9aa28a81ec CircleCI: Automatically detect and repeat new or modified JUnit tests
9aa28a81ec is described below
commit 9aa28a81ecb65d8e586039f419ca3272c1de64c2
Author: Andrés de la Peña <a....@gmail.com>
AuthorDate: Sat Oct 15 15:38:37 2022 +0100
CircleCI: Automatically detect and repeat new or modified JUnit tests
patch by Andrés de la Peña; reviewed by Josh McKenzie for CASSANDRA-17939
---
.circleci/config-2_1.yml | 544 ++++++--
.circleci/config-2_1.yml.high_res.patch | 12 +-
.circleci/config-2_1.yml.mid_res.patch | 20 +-
.circleci/config.yml | 2190 +++++++++++++++++++----------
.circleci/config.yml.HIGHRES | 2287 +++++++++++++++++++++----------
.circleci/config.yml.LOWRES | 2287 +++++++++++++++++++++----------
.circleci/config.yml.MIDRES | 2161 ++++++++++++++++++++---------
.circleci/generate.sh | 141 +-
.circleci/readme.md | 101 +-
build.xml | 15 +-
10 files changed, 6765 insertions(+), 2993 deletions(-)
diff --git a/.circleci/config-2_1.yml b/.circleci/config-2_1.yml
index 35eb49206e..0b22f29349 100644
--- a/.circleci/config-2_1.yml
+++ b/.circleci/config-2_1.yml
@@ -43,59 +43,77 @@ default_env_vars: &default_env_vars
CCM_MAX_HEAP_SIZE: 1024M
CCM_HEAP_NEWSIZE: 256M
+ # Whether the repeated test iterations should stop on the first failure by default.
+ REPEATED_TESTS_STOP_ON_FAILURE: false
+
+ # Comma-separated list of tests that should be included in the repeated run for regular unit tests,
+ # in addition to automatically detected new and modified tests. For example:
+ # REPEATED_UTESTS: org.apache.cassandra.cql3.ViewTest
+ # REPEATED_UTESTS: org.apache.cassandra.cql3.ViewTest#testCountersTable
+ # REPEATED_UTESTS: org.apache.cassandra.cql3.ViewTest,org.apache.cassandra.cql3.functions.TimeFctsTest
+ REPEATED_UTESTS:
+ # The number of times that new, modified or manually specified unit tests should be run.
+ REPEATED_UTESTS_COUNT: 500
+
+ # Comma-separated list of tests that should be included in the repeated run for long unit tests,
+ # in addition to automatically detected new and modified tests. For example:
+ # REPEATED_UTESTS_LONG: org.apache.cassandra.db.commitlog.CommitLogStressTest
+ # REPEATED_UTESTS_LONG: org.apache.cassandra.db.commitlog.CommitLogStressTest#testRandomSize
+ REPEATED_UTESTS_LONG:
+ # The number of times that new, modified or manually specified long unit tests should be run.
+ REPEATED_UTESTS_LONG_COUNT: 100
+
+ # Comma-separated list of tests that should be included in the repeated run for JVM dtests,
+ # in addition to automatically detected new and modified tests. For example:
+ # REPEATED_JVM_DTESTS: org.apache.cassandra.distributed.test.PagingTest
+ # REPEATED_JVM_DTESTS: org.apache.cassandra.distributed.test.PagingTest#testPaging
+ REPEATED_JVM_DTESTS:
+ # The number of times that new, modified or manually specified JVM dtests should be run.
+ REPEATED_JVM_DTESTS_COUNT: 500
+
+ # Comma-separated list of tests that should be included in the repeated run for JVM upgrade dtests,
+ # in addition to automatically detected new and modified tests. For example:
+ # REPEATED_JVM_UPGRADE_DTESTS: org.apache.cassandra.distributed.upgrade.GroupByTest
+ # REPEATED_JVM_UPGRADE_DTESTS: org.apache.cassandra.distributed.upgrade.GroupByTest#testReads
+ REPEATED_JVM_UPGRADE_DTESTS:
+ # The number of times that new, modified or manually specified JVM upgrade dtests should be run.
+ REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+
+ # Comma-separated list of Python dtests that should be repeatedly run, for example:
+ # REPEATED_DTESTS: cqlsh_tests/test_cqlsh.py
+ # REPEATED_DTESTS: cqlsh_tests/test_cqlsh.py::TestCqlshSmoke
+ # REPEATED_DTESTS: cqlsh_tests/test_cqlsh.py::TestCqlshSmoke::test_create_index
+ # REPEATED_DTESTS: cqlsh_tests/test_cqlsh.py,consistency_test.py
+ REPEATED_DTESTS:
+ # The number of times that the manually specified Python dtests should be run.
+ REPEATED_DTESTS_COUNT: 500
+
+ # Comma-separated list of Python dtests that should be repeatedly run, for example:
+ # REPEATED_UPGRADE_DTESTS: upgrade_tests/cql_tests.py
+ # REPEATED_UPGRADE_DTESTS: upgrade_tests/cql_tests.py::TestCQLNodes2RF1_Upgrade_current_4_0_x_To_indev_4_0_x
+ # REPEATED_UPGRADE_DTESTS: upgrade_tests/cql_tests.py,upgrade_tests/paging_test.py
+ REPEATED_UPGRADE_DTESTS:
+ # The number of times that the manually specified Python upgrade dtests should be run.
+ REPEATED_UPGRADE_DTESTS_COUNT: 25
+
# The Ant test target to run, for example:
- # REPEATED_UTEST_TARGET: testsome
- # REPEATED_UTEST_TARGET: test-jvm-dtest-some
- # REPEATED_UTEST_TARGET: test-cdc
- # REPEATED_UTEST_TARGET: test-compression
- # REPEATED_UTEST_TARGET: test-system-keyspace-directory
- REPEATED_UTEST_TARGET: testsome
+ # REPEATED_ANT_TEST_TARGET: testsome
+ # REPEATED_ANT_TEST_TARGET: test-jvm-dtest-some
+ # REPEATED_ANT_TEST_TARGET: test-cdc
+ # REPEATED_ANT_TEST_TARGET: test-compression
+ # REPEATED_ANT_TEST_TARGET: test-system-keyspace-directory
+ REPEATED_ANT_TEST_TARGET: testsome
# The name of JUnit class to be run multiple times, for example:
- # REPEATED_UTEST_CLASS: org.apache.cassandra.cql3.ViewTest
- # REPEATED_UTEST_CLASS: org.apache.cassandra.distributed.test.PagingTest
- REPEATED_UTEST_CLASS:
- # The optional specific methods within REPEATED_UTEST_CLASS to be run, for example:
- # REPEATED_UTEST_METHODS: testCompoundPartitionKey
- # REPEATED_UTEST_METHODS: testCompoundPartitionKey,testStaticTable
+ # REPEATED_ANT_TEST_CLASS: org.apache.cassandra.cql3.ViewTest
+ # REPEATED_ANT_TEST_CLASS: org.apache.cassandra.distributed.test.PagingTest
+ REPEATED_ANT_TEST_CLASS:
+ # The optional specific methods within REPEATED_ANT_TEST_CLASS to be run, for example:
+ # REPEATED_ANT_TEST_METHODS: testCompoundPartitionKey
+ # REPEATED_ANT_TEST_METHODS: testCompoundPartitionKey,testStaticTable
# Please note that some Ant targets will ignore the -Dtest.methods argument produced by this.
- REPEATED_UTEST_METHODS:
- # The number of times that the repeated JUnit test should be run
- REPEATED_UTEST_COUNT: 100
- # Whether the test iteration should stop on the first failure
- REPEATED_UTEST_STOP_ON_FAILURE: false
-
- # A Python dtest to be run multiple times, for example:
- # REPEATED_DTEST_NAME: cqlsh_tests/test_cqlsh.py
- # REPEATED_DTEST_NAME: cqlsh_tests/test_cqlsh.py::TestCqlshSmoke
- # REPEATED_DTEST_NAME: cqlsh_tests/test_cqlsh.py::TestCqlshSmoke::test_create_index
- REPEATED_DTEST_NAME:
- # Whether the repeated Python dtest should use vnodes
- REPEATED_DTEST_VNODES: false
- # The number of times that the repeated Python dtest should be run
- REPEATED_DTEST_COUNT: 100
- # Whether the test iteration should stop on the first failure
- REPEATED_DTEST_STOP_ON_FAILURE: false
-
- # A Python upgrade dtest to be run multiple times, for example:
- # REPEATED_UPGRADE_DTEST_NAME: upgrade_tests/cql_tests.py
- # REPEATED_UPGRADE_DTEST_NAME: upgrade_tests/cql_tests.py::TestCQLNodes2RF1_Upgrade_current_4_0_x_To_indev_4_0_x
- REPEATED_UPGRADE_DTEST_NAME:
- # The number of times that the repeated Python upgrade dtest should be run
- REPEATED_UPGRADE_DTEST_COUNT: 100
- # Whether the test iteration should stop on the first failure
- REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
-
- # The name of JVM upgrade dtest class to be run multiple times, for example:
- # REPEATED_JVM_UPGRADE_DTEST_CLASS: org.apache.cassandra.distributed.upgrade.MixedModeAvailabilityV30Test
- REPEATED_JVM_UPGRADE_DTEST_CLASS:
- # The optional specific methods within REPEATED_JVM_UPGRADE_DTEST_CLASS to be run, for example:
- # REPEATED_JVM_UPGRADE_DTEST_METHODS: testAvailabilityV30ToV3X
- # REPEATED_JVM_UPGRADE_DTEST_METHODS: testAvailabilityV30ToV3X,testAvailabilityV30ToV4
- REPEATED_JVM_UPGRADE_DTEST_METHODS:
- # The number of times that the repeated JVM upgrade dtest should be run
- REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- # Whether the JVM upgrade dtest iteration should stop on the first failure
- REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ REPEATED_ANT_TEST_METHODS:
+ # The number of times that the repeated JUnit test should be run.
+ REPEATED_ANT_TEST_COUNT: 500
j8_par_executor: &j8_par_executor
executor:
@@ -141,7 +159,7 @@ j8_repeated_jvm_upgrade_dtest_executor: &j8_repeated_jvm_upgrade_dtest_executor
name: java8-executor
parallelism: 4
-with_dtests_jobs: &with_dtest_jobs
+separate_jobs: &separate_jobs
jobs:
- start_build:
type: approval
@@ -155,12 +173,24 @@ with_dtests_jobs: &with_dtest_jobs
requires:
- start_j8_unit_tests
- build
+ - start_j8_unit_tests_repeat:
+ type: approval
+ - j8_unit_tests_repeat:
+ requires:
+ - start_j8_unit_tests_repeat
+ - build
- start_j8_jvm_dtests:
type: approval
- j8_jvm_dtests:
requires:
- start_j8_jvm_dtests
- build
+ - start_j8_jvm_dtests_repeat:
+ type: approval
+ - j8_jvm_dtests_repeat:
+ requires:
+ - start_j8_jvm_dtests_repeat
+ - build
# specialized unit tests (all run using Java 8)
- start_utests_long:
type: approval
@@ -168,70 +198,88 @@ with_dtests_jobs: &with_dtest_jobs
requires:
- start_utests_long
- build
+ - start_utests_long_repeat:
+ type: approval
+ - utests_long_repeat:
+ requires:
+ - start_utests_long_repeat
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
+ - start_utests_compression_repeat:
+ type: approval
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression_repeat
+ - build
- start_j8_dtest_jars_build:
type: approval
- j8_dtest_jars_build:
requires:
- start_j8_dtest_jars_build
- build
- - start_jvm_upgrade_dtest:
+ - start_jvm_upgrade_dtests:
type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
+ - start_jvm_upgrade_dtests
+ - j8_dtest_jars_build
+ - start_jvm_upgrade_dtests_repeat:
+ type: approval
+ - j8_jvm_upgrade_dtests_repeat:
+ requires:
+ - start_jvm_upgrade_dtests_repeat
- j8_dtest_jars_build
# Java 8 dtests
- start_j8_dtests:
type: approval
- - j8_dtests-with-vnodes:
+ - j8_dtests:
requires:
- start_j8_dtests
- build
- - j8_dtests-no-vnodes:
+ - start_j8_dtests_repeat:
+ type: approval
+ - j8_dtests_repeat:
requires:
- - start_j8_dtests
+ - start_j8_dtests_repeat
- build
- # Java 8 upgrade tests
- - start_upgrade_tests:
+ - start_j8_dtests_vnode:
type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_vnode:
requires:
- - start_upgrade_tests
+ - start_j8_dtests_vnode
- build
- # Java 8 repeated utest
- - start_j8_repeated_utest:
+ - start_j8_dtests_vnode_repeat:
type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_utest
+ - start_j8_dtests_vnode_repeat
- build
- # Java 8 repeated dtest
- - start_j8_repeated_dtest:
+ # Java 8 upgrade tests
+ - start_j8_upgrade_dtests:
type: approval
- - j8_repeated_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_j8_repeated_dtest
+ - start_j8_upgrade_dtests
- build
- # Repeated Python upgrade dtest
- - start_repeated_upgrade_dtest:
+ - start_j8_upgrade_dtests_repeat:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_upgrade_dtest
+ - start_j8_upgrade_dtests_repeat
- build
- # Repeated JVM upgrade dtest
- - start_repeated_jvm_upgrade_dtest:
+ # Java 8 repeated utest
+ - start_j8_repeated_ant_test:
type: approval
- - repeated_jvm_upgrade_dtest:
+ - j8_repeated_ant_test:
requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
+ - start_j8_repeated_ant_test
+ - build
+
pre-commit_jobs: &pre-commit_jobs
jobs:
@@ -244,9 +292,15 @@ pre-commit_jobs: &pre-commit_jobs
- j8_unit_tests:
requires:
- build
+ - j8_unit_tests_repeat:
+ requires:
+ - build
- j8_jvm_dtests:
requires:
- build
+ - j8_jvm_dtests_repeat:
+ requires:
+ - build
# specialized unit tests (all run on request using Java 8)
- start_utests_long:
type: approval
@@ -254,70 +308,60 @@ pre-commit_jobs: &pre-commit_jobs
requires:
- start_utests_long
- build
+ - utests_long_repeat:
+ requires:
+ - start_utests_long
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
- - start_j8_dtest_jars_build:
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression
+ - build
+ - start_jvm_upgrade_dtests:
type: approval
- j8_dtest_jars_build:
requires:
- - start_j8_dtest_jars_build
+ - start_jvm_upgrade_dtests
- build
- - start_jvm_upgrade_dtest:
- type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
+ - j8_dtest_jars_build
+ - j8_jvm_upgrade_dtests_repeat:
+ requires:
- j8_dtest_jars_build
# Java 8 dtests
- - j8_dtests-with-vnodes:
+ - j8_dtests:
requires:
- build
- - j8_dtests-no-vnodes:
+ - j8_dtests_repeat:
requires:
- build
- # Java 8 upgrade tests (on request)
- - start_upgrade_tests:
- type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_vnode:
requires:
- - start_upgrade_tests
- build
- # Java 8 repeated utest (on request)
- - start_j8_repeated_utest:
- type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_utest
- build
- # Java 8 repeated dtest (on request)
- - start_j8_repeated_dtest:
+ # Java 8 upgrade tests (on request)
+ - start_upgrade_dtests:
type: approval
- - j8_repeated_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_j8_repeated_dtest
+ - start_upgrade_dtests
- build
- # Repeated Python upgrade dtest (on request)
- - start_repeated_upgrade_dtest:
- type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_upgrade_dtest
+ - start_upgrade_dtests
- build
- # Repeated JVM upgrade dtest (on request)
- - start_repeated_jvm_upgrade_dtest:
- type: approval
- - repeated_jvm_upgrade_dtest:
- requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
workflows:
version: 2
- separate_tests: *with_dtest_jobs
+ separate_tests: *separate_jobs
pre-commit_tests: *pre-commit_jobs
executors:
@@ -414,7 +458,7 @@ jobs:
- run_parallel_junit_tests:
target: testclasslist-compression
- j8_dtests-with-vnodes:
+ j8_dtests_vnode:
<<: *j8_par_executor
steps:
- attach_workspace:
@@ -428,7 +472,7 @@ jobs:
file_tag: j8_with_vnodes
pytest_extra_args: '--use-vnodes --num-tokens=32 --skip-resource-intensive-tests'
- j8_dtests-no-vnodes:
+ j8_dtests:
<<: *j8_par_executor
steps:
- attach_workspace:
@@ -442,7 +486,7 @@ jobs:
file_tag: j8_without_vnodes
pytest_extra_args: '--skip-resource-intensive-tests'
- j8_upgradetests-no-vnodes:
+ j8_upgrade_dtests:
<<: *j8_par_executor
steps:
- attach_workspace:
@@ -459,20 +503,68 @@ jobs:
extra_env_args: 'RUN_STATIC_UPGRADE_MATRIX=true'
pytest_extra_args: '--execute-upgrade-tests'
- j8_repeated_utest:
+ j8_unit_tests_repeat:
+ <<: *j8_repeated_utest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_unit_tests_repeat
+
+ utests_compression_repeat:
+ <<: *j8_repeated_utest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_utests_compression_repeat
+
+ utests_system_keyspace_directory_repeat:
+ <<: *j8_repeated_utest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_utests_system_keyspace_directory_repeat
+
+ utests_long_repeat:
+ <<: *j8_repeated_utest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_utests_long_repeat
+
+ j8_jvm_dtests_repeat:
+ <<: *j8_repeated_utest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_jvm_dtests_repeat
+
+ j8_jvm_upgrade_dtests_repeat:
+ <<: *j8_repeated_jvm_upgrade_dtest_executor
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - log_environment
+ - run_jvm_upgrade_dtests_repeat
+
+ j8_repeated_ant_test:
<<: *j8_repeated_utest_executor
steps:
- attach_workspace:
at: /home/cassandra
- log_environment
- run_repeated_utest:
- target: ${REPEATED_UTEST_TARGET}
- class: ${REPEATED_UTEST_CLASS}
- methods: ${REPEATED_UTEST_METHODS}
- count: ${REPEATED_UTEST_COUNT}
- stop_on_failure: ${REPEATED_UTEST_STOP_ON_FAILURE}
+ target: ${REPEATED_ANT_TEST_TARGET}
+ class: ${REPEATED_ANT_TEST_CLASS}
+ methods: ${REPEATED_ANT_TEST_METHODS}
+ count: ${REPEATED_ANT_TEST_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
- j8_repeated_dtest:
+ j8_dtests_repeat:
<<: *j8_repeated_dtest_executor
steps:
- attach_workspace:
@@ -480,26 +572,27 @@ jobs:
- clone_dtest
- create_venv
- run_repeated_dtest:
- tests: ${REPEATED_DTEST_NAME}
- vnodes: ${REPEATED_DTEST_VNODES}
+ tests: ${REPEATED_DTESTS}
+ vnodes: "false"
upgrade: "false"
- count: ${REPEATED_DTEST_COUNT}
- stop_on_failure: ${REPEATED_DTEST_STOP_ON_FAILURE}
+ count: ${REPEATED_DTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
- repeated_jvm_upgrade_dtest:
- <<: *j8_repeated_jvm_upgrade_dtest_executor
+ j8_dtests_vnode_repeat:
+ <<: *j8_repeated_dtest_executor
steps:
- attach_workspace:
at: /home/cassandra
- - log_environment
- - run_repeated_utest:
- target: test-jvm-dtest-some
- class: ${REPEATED_JVM_UPGRADE_DTEST_CLASS}
- methods: ${REPEATED_JVM_UPGRADE_DTEST_METHODS}
- count: ${REPEATED_JVM_UPGRADE_DTEST_COUNT}
- stop_on_failure: ${REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE}
+ - clone_dtest
+ - create_venv
+ - run_repeated_dtest:
+ tests: ${REPEATED_DTESTS}
+ vnodes: "true"
+ upgrade: "false"
+ count: ${REPEATED_DTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
- repeated_upgrade_dtest:
+ j8_upgrade_dtests_repeat:
<<: *j8_repeated_upgrade_dtest_executor
steps:
- attach_workspace:
@@ -507,11 +600,11 @@ jobs:
- clone_dtest
- create_venv
- run_repeated_dtest:
- tests: ${REPEATED_UPGRADE_DTEST_NAME}
+ tests: ${REPEATED_UPGRADE_DTESTS}
vnodes: "false"
upgrade: "true"
- stop_on_failure: ${REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE}
- count: ${REPEATED_UPGRADE_DTEST_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+ count: ${REPEATED_UPGRADE_DTESTS_COUNT}
commands:
log_environment:
@@ -831,6 +924,172 @@ commands:
path: ~/cassandra-dtest/logs
destination: dtest_<<parameters.file_tag>>_logs
+ run_unit_tests_repeat:
+ steps:
+ - run_repeated_utests:
+ target: testsome
+ tests: ${REPEATED_UTESTS}
+ count: ${REPEATED_UTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_utests_compression_repeat:
+ steps:
+ - run_repeated_utests:
+ target: test-compression
+ tests: ${REPEATED_UTESTS}
+ count: ${REPEATED_UTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_utests_system_keyspace_directory_repeat:
+ steps:
+ - run_repeated_utests:
+ target: test-system-keyspace-directory
+ tests: ${REPEATED_UTESTS}
+ count: ${REPEATED_UTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_utests_long_repeat:
+ steps:
+ - run_repeated_utests:
+ target: long-testsome
+ tests: ${REPEATED_UTESTS_LONG}
+ count: ${REPEATED_UTESTS_LONG_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_jvm_dtests_repeat:
+ steps:
+ - run_repeated_utests:
+ target: test-jvm-dtest-some
+ tests: ${REPEATED_JVM_DTESTS}
+ count: ${REPEATED_JVM_DTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_jvm_upgrade_dtests_repeat:
+ steps:
+ - run_repeated_utests:
+ target: test-jvm-dtest-some
+ tests: ${REPEATED_JVM_UPGRADE_DTESTS}
+ count: ${REPEATED_JVM_UPGRADE_DTESTS_COUNT}
+ stop_on_failure: ${REPEATED_TESTS_STOP_ON_FAILURE}
+
+ run_repeated_utests:
+ parameters:
+ target:
+ type: string
+ tests:
+ type: string
+ count:
+ type: string
+ stop_on_failure:
+ type: string
+ steps:
+ - run:
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((<<parameters.count>> / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (<<parameters.count>> % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo <<parameters.tests>> | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant <<parameters.target>> $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ <<parameters.stop_on_failure>> = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+
run_repeated_utest:
parameters:
target:
@@ -885,7 +1144,8 @@ commands:
if [[ $target == "test" || \
$target == "test-cdc" || \
$target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
name="-Dtest.name=$class_name"
else
name="-Dtest.name=$class_path"
@@ -1002,6 +1262,8 @@ commands:
echo "** done env"
mkdir -p /tmp/results/dtests
+ tests_arg=$(echo <<parameters.tests>> | sed -e "s/,/ /g")
+
stop_on_failure_arg=""
if <<parameters.stop_on_failure>>; then
stop_on_failure_arg="-x"
@@ -1014,11 +1276,11 @@ commands:
upgrade_arg=""
if <<parameters.upgrade>>; then
- upgrade_arg="--execute-upgrade-tests"
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
fi
# we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir <<parameters.tests>> | tee /tmp/dtest/stdout.txt
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
fi
fi
- store_test_results:
diff --git a/.circleci/config-2_1.yml.high_res.patch b/.circleci/config-2_1.yml.high_res.patch
index 9bb0d6104e..bdc37a0307 100644
--- a/.circleci/config-2_1.yml.high_res.patch
+++ b/.circleci/config-2_1.yml.high_res.patch
@@ -1,5 +1,5 @@
---- .circleci/config-2_1.yml 2021-10-19 11:08:44.000000000 +0100
-+++ .circleci/config-2_1.yml.HIGHRES 2021-10-19 11:26:16.000000000 +0100
+--- config-2_1.yml 2022-10-15 16:29:48.670020291 +0100
++++ config-2_1.yml.HIGHRES 2022-10-15 16:30:17.885329362 +0100
@@ -40,8 +40,8 @@
CASSANDRA_SKIP_SYNC: true
DTEST_REPO: https://github.com/apache/cassandra-dtest.git
@@ -9,9 +9,9 @@
+ CCM_MAX_HEAP_SIZE: 2048M
+ CCM_HEAP_NEWSIZE: 512M
- # The Ant test target to run, for example:
- # REPEATED_UTEST_TARGET: testsome
-@@ -100,46 +100,50 @@
+ # Whether the repeated test iterations should stop on the first failure by default.
+ REPEATED_TESTS_STOP_ON_FAILURE: false
+@@ -118,46 +118,50 @@
j8_par_executor: &j8_par_executor
executor:
name: java8-executor
@@ -71,5 +71,5 @@
+ exec_resource_class: xlarge
+ parallelism: 100
- with_dtests_jobs: &with_dtest_jobs
+ separate_jobs: &separate_jobs
jobs:
diff --git a/.circleci/config-2_1.yml.mid_res.patch b/.circleci/config-2_1.yml.mid_res.patch
index bcb47fcaff..05cf0c275e 100644
--- a/.circleci/config-2_1.yml.mid_res.patch
+++ b/.circleci/config-2_1.yml.mid_res.patch
@@ -1,6 +1,6 @@
---- .circleci/config-2_1.yml 2021-10-19 11:08:44.000000000 +0100
-+++ .circleci/config-2_1.yml.MIDRES 2021-10-19 11:26:16.000000000 +0100
-@@ -101,45 +101,65 @@
+--- config-2_1.yml 2022-10-15 16:29:48.670020291 +0100
++++ config-2_1.yml.MIDRES 2022-10-15 16:30:17.879007207 +0100
+@@ -119,45 +119,65 @@
executor:
name: java8-executor
#exec_resource_class: xlarge
@@ -71,30 +71,30 @@
+ exec_resource_class: medium
+ parallelism: 25
- with_dtests_jobs: &with_dtest_jobs
+ separate_jobs: &separate_jobs
jobs:
-@@ -415,7 +435,7 @@
+@@ -459,7 +479,7 @@
target: testclasslist-compression
- j8_dtests-with-vnodes:
+ j8_dtests_vnode:
- <<: *j8_par_executor
+ <<: *j8_large_par_executor
steps:
- attach_workspace:
at: /home/cassandra
-@@ -429,7 +449,7 @@
+@@ -473,7 +493,7 @@
pytest_extra_args: '--use-vnodes --num-tokens=32 --skip-resource-intensive-tests'
- j8_dtests-no-vnodes:
+ j8_dtests:
- <<: *j8_par_executor
+ <<: *j8_large_par_executor
steps:
- attach_workspace:
at: /home/cassandra
-@@ -443,7 +463,7 @@
+@@ -487,7 +507,7 @@
pytest_extra_args: '--skip-resource-intensive-tests'
- j8_upgradetests-no-vnodes:
+ j8_upgrade_dtests:
- <<: *j8_par_executor
+ <<: *j8_very_large_par_executor
steps:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index a3fd9e1a2d..43a507ba9f 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -108,25 +108,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_jvm_upgrade_dtest:
+ utests_compression_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -159,81 +160,80 @@ jobs:
which java
java -version
- run:
- name: Run repeated JUnit test
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_JVM_UPGRADE_DTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_JVM_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running test-jvm-dtest-some ${REPEATED_JVM_UPGRADE_DTEST_CLASS} ${REPEATED_JVM_UPGRADE_DTEST_METHODS} ${REPEATED_JVM_UPGRADE_DTEST_COUNT} times"
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- target=test-jvm-dtest-some
- class_path=${REPEATED_JVM_UPGRADE_DTEST_CLASS}
- class_name="${class_path##*.}"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
# Prepare the -Dtest.name argument.
# It can be the fully qualified class name or the short class name, depending on the target.
if [[ $target == "test" || \
$target == "test-cdc" || \
$target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
else
- name="-Dtest.name=$class_path"
+ name_arg="-Dtest.name=$class"
fi
# Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_JVM_UPGRADE_DTEST_METHODS}" == "<nil>" ]; then
- methods=""
+ if [ $method == "" ]; then
+ methods_arg=""
else
- methods="-Dtest.methods=${REPEATED_JVM_UPGRADE_DTEST_METHODS}"
+ methods_arg="-Dtest.methods=$method"
fi
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
+ echo "Running test $test, iteration $i of $count"
# run the test
status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ if !( set -o pipefail && \
+ ant test-compression $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
status="fails"
exit_code=1
fi
# move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
mkdir -p $dest
- mv stdout.txt $dest/test-jvm-dtest-some-${REPEATED_JVM_UPGRADE_DTEST_CLASS}.txt
+ mv stdout.txt $dest/${test}.txt
# move the XML output files
source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
@@ -241,31 +241,29 @@ jobs:
# move the log files
source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
fi
# maybe stop iterations on test failure
- if [[ ${REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
break
fi
done
-
- (exit ${exit_code})
- fi
- fi
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/results/repeated_utest/stdout
+ path: /tmp/results/repeated_utests/stdout
destination: stdout
- store_artifacts:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
destination: junitxml
- store_artifacts:
- path: /tmp/results/repeated_utest/logs
+ path: /tmp/results/repeated_utests/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -282,25 +280,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_upgrade_dtest:
+ utests_long_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -311,82 +310,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_UPGRADE_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UPGRADE_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_LONG_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_LONG_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS_LONG} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if false; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if true; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_UPGRADE_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant long-testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -402,25 +452,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_dtest:
+ j8_unit_tests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -431,77 +482,251 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if ${REPEATED_DTEST_VNODES}; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if false; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
- - store_test_results:
- path: /tmp/results
- - store_artifacts:
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_UPGRADE_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_UPGRADE_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_UPGRADE_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if true; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
path: /tmp/dtest
destination: dtest
- store_artifacts:
@@ -522,22 +747,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
build:
@@ -619,25 +845,546 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_dtests_vnode_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if true; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Determine Tests to Run (j8_upgradetests_without_vnodes)
+ no_output_timeout: 5m
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ cd cassandra-dtest
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
+ set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
+ if [ -z '^upgrade_tests' ]; then
+ mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
+ else
+ grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
+ fi
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ - run:
+ name: Run dtests (j8_upgradetests_without_vnodes)
+ no_output_timeout: 15m
+ command: |
+ echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
+ set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest_j8_upgradetests_without_vnodes
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_j8_upgradetests_without_vnodes_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_jvm_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_UPGRADE_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_unit_tests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Determine unit Tests to Run
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ rm -fr ~/cassandra-dtest/upgrade_tests
+ echo "***java tests***"
+
+ # get all of our unit test filenames
+ set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
+
+ # split up the unit tests into groups based on the number of containers we have
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
+ set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
+ cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ no_output_timeout: 15m
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Run Unit Tests (testclasslist)
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
+ if [ -z "$test_timeout" ]; then
+ test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ fi
+ ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
+ no_output_timeout: 15m
+ - store_test_results:
+ path: /tmp/cassandra/build/test/output/
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-no-vnodes:
+ j8_dtests:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -693,25 +1440,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_upgradetests-no-vnodes:
+ j8_dtests_vnode:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -737,62 +1485,21 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_upgradetests_without_vnodes)
+ name: Determine Tests to Run (j8_with_vnodes)
no_output_timeout: 5m
- command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- cd cassandra-dtest
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
-
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
- fi
-
- echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
- set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
- if [ -z '^upgrade_tests' ]; then
- mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
- else
- grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
- fi
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- run:
- name: Run dtests (j8_upgradetests_without_vnodes)
+ name: Run dtests (j8_with_vnodes)
no_output_timeout: 15m
- command: |
- echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
-
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
- fi
-
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
-
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
- set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_upgradetests_without_vnodes
+ destination: dtest_j8_with_vnodes
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_upgradetests_without_vnodes_logs
+ destination: dtest_j8_with_vnodes_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -808,25 +1515,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_unit_tests:
+ j8_jvm_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -836,25 +1544,6 @@ jobs:
steps:
- attach_workspace:
at: /home/cassandra
- - run:
- name: Determine unit Tests to Run
- command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- rm -fr ~/cassandra-dtest/upgrade_tests
- echo "***java tests***"
-
- # get all of our unit test filenames
- set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
-
- # split up the unit tests into groups based on the number of containers we have
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
- set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
- cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- no_output_timeout: 15m
- run:
name: Log Environment Information
command: |
@@ -878,7 +1567,8 @@ jobs:
which java
java -version
- run:
- name: Run Unit Tests (testclasslist)
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
command: |
set -x
export PATH=$JAVA_HOME/bin:$PATH
@@ -887,19 +1577,276 @@ jobs:
if [ -d ~/dtest_jars ]; then
cp ~/dtest_jars/dtest* /tmp/cassandra/build/
fi
- test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
- if [ -z "$test_timeout" ]; then
- test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_repeated_ant_test:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Run repeated JUnit test
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_ANT_TEST_CLASS}" == "<nil>" ]; then
+ echo "Repeated utest class name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" == "<nil>" ]; then
+ echo "Repeated utest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" -le 0 ]; then
+ echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_ANT_TEST_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_ANT_TEST_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_ANT_TEST_TARGET} ${REPEATED_ANT_TEST_CLASS} ${REPEATED_ANT_TEST_METHODS} ${REPEATED_ANT_TEST_COUNT} times"
+
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ target=${REPEATED_ANT_TEST_TARGET}
+ class_path=${REPEATED_ANT_TEST_CLASS}
+ class_name="${class_path##*.}"
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name="-Dtest.name=$class_name"
+ else
+ name="-Dtest.name=$class_path"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ "${REPEATED_ANT_TEST_METHODS}" == "<nil>" ]; then
+ methods=""
+ else
+ methods="-Dtest.methods=${REPEATED_ANT_TEST_METHODS}"
+ fi
+
+ # Run the test target as many times as requested collecting the exit code,
+ # stopping the iteration only if stop_on_failure is set.
+ exit_code="$?"
+ for i in $(seq -w 1 $count); do
+
+ echo "Running test iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${REPEATED_ANT_TEST_TARGET}-${REPEATED_ANT_TEST_CLASS}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utest/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+
+ (exit ${exit_code})
+ fi
fi
- ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
- no_output_timeout: 15m
- store_test_results:
- path: /tmp/cassandra/build/test/output/
+ path: /tmp/results/repeated_utest/output
- store_artifacts:
- path: /tmp/cassandra/build/test/output
+ path: /tmp/results/repeated_utest/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utest/output
destination: junitxml
- store_artifacts:
- path: /tmp/cassandra/build/test/logs
+ path: /tmp/results/repeated_utest/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -916,25 +1863,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-with-vnodes:
+ j8_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -960,21 +1908,69 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_with_vnodes)
- no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- - run:
- name: Run dtests (j8_with_vnodes)
+ name: Run repeated Python dtest
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_with_vnodes
+ destination: dtest
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_with_vnodes_logs
+ destination: dtest_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -990,22 +1986,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_jvm_dtests:
@@ -1098,22 +2095,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_long:
@@ -1160,22 +2158,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_compression:
@@ -1268,196 +2267,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_utest:
- docker:
- - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
- resource_class: medium
- working_directory: ~/
- shell: /bin/bash -eo pipefail -l
- parallelism: 4
- steps:
- - attach_workspace:
- at: /home/cassandra
- - run:
- name: Log Environment Information
- command: |
- echo '*** id ***'
- id
- echo '*** cat /proc/cpuinfo ***'
- cat /proc/cpuinfo
- echo '*** free -m ***'
- free -m
- echo '*** df -m ***'
- df -m
- echo '*** ifconfig -a ***'
- ifconfig -a
- echo '*** uname -a ***'
- uname -a
- echo '*** mount ***'
- mount
- echo '*** env ***'
- env
- echo '*** java ***'
- which java
- java -version
- - run:
- name: Run repeated JUnit test
- no_output_timeout: 15m
- command: |
- if [ "${REPEATED_UTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UTEST_TARGET} ${REPEATED_UTEST_CLASS} ${REPEATED_UTEST_METHODS} ${REPEATED_UTEST_COUNT} times"
-
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
-
- target=${REPEATED_UTEST_TARGET}
- class_path=${REPEATED_UTEST_CLASS}
- class_name="${class_path##*.}"
-
- # Prepare the -Dtest.name argument.
- # It can be the fully qualified class name or the short class name, depending on the target.
- if [[ $target == "test" || \
- $target == "test-cdc" || \
- $target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
- else
- name="-Dtest.name=$class_path"
- fi
-
- # Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_UTEST_METHODS}" == "<nil>" ]; then
- methods=""
- else
- methods="-Dtest.methods=${REPEATED_UTEST_METHODS}"
- fi
-
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
- for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
-
- # run the test
- status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
- status="fails"
- exit_code=1
- fi
-
- # move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
- mkdir -p $dest
- mv stdout.txt $dest/${REPEATED_UTEST_TARGET}-${REPEATED_UTEST_CLASS}.txt
-
- # move the XML output files
- source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # move the log files
- source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # maybe stop iterations on test failure
- if [[ ${REPEATED_UTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
- break
- fi
- done
-
- (exit ${exit_code})
- fi
- fi
- - store_test_results:
- path: /tmp/results/repeated_utest/output
- - store_artifacts:
- path: /tmp/results/repeated_utest/stdout
- destination: stdout
- - store_artifacts:
- path: /tmp/results/repeated_utest/output
- destination: junitxml
- - store_artifacts:
- path: /tmp/results/repeated_utest/logs
- destination: logs
- environment:
- - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - ANT_HOME: /usr/share/ant
- - LANG: en_US.UTF-8
- - KEEP_TEST_DIR: true
- - DEFAULT_DIR: /home/cassandra/cassandra-dtest
- - PYTHONIOENCODING: utf-8
- - PYTHONUNBUFFERED: true
- - CASS_DRIVER_NO_EXTENSIONS: true
- - CASS_DRIVER_NO_CYTHON: true
- - CASSANDRA_SKIP_SYNC: true
- - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: trunk
- - CCM_MAX_HEAP_SIZE: 1024M
- - CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_dtest_jars_build:
@@ -1536,22 +2362,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
workflows:
@@ -1593,52 +2420,30 @@ workflows:
requires:
- start_j8_dtest_jars_build
- build
- - start_jvm_upgrade_dtest:
+ - start_jvm_upgrade_dtests:
type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
+ - start_jvm_upgrade_dtests
- j8_dtest_jars_build
- start_j8_dtests:
type: approval
- - j8_dtests-with-vnodes:
- requires:
- - start_j8_dtests
- - build
- - j8_dtests-no-vnodes:
+ - j8_dtests:
requires:
- start_j8_dtests
- build
- - start_upgrade_tests:
+ - start_j8_dtests_vnode:
type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_vnode:
requires:
- - start_upgrade_tests
+ - start_j8_dtests_vnode
- build
- - start_j8_repeated_utest:
+ - start_j8_upgrade_dtests:
type: approval
- - j8_repeated_utest:
+ - j8_upgrade_dtests:
requires:
- - start_j8_repeated_utest
+ - start_j8_upgrade_dtests
- build
- - start_j8_repeated_dtest:
- type: approval
- - j8_repeated_dtest:
- requires:
- - start_j8_repeated_dtest
- - build
- - start_repeated_upgrade_dtest:
- type: approval
- - repeated_upgrade_dtest:
- requires:
- - start_repeated_upgrade_dtest
- - build
- - start_repeated_jvm_upgrade_dtest:
- type: approval
- - repeated_jvm_upgrade_dtest:
- requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
pre-commit_tests:
jobs:
- start_pre-commit_tests:
@@ -1664,51 +2469,24 @@ workflows:
requires:
- start_utests_compression
- build
- - start_j8_dtest_jars_build:
+ - start_jvm_upgrade_dtests:
type: approval
- j8_dtest_jars_build:
requires:
- - start_j8_dtest_jars_build
+ - start_jvm_upgrade_dtests
- build
- - start_jvm_upgrade_dtest:
- type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
- j8_dtest_jars_build
- - j8_dtests-with-vnodes:
- requires:
- - build
- - j8_dtests-no-vnodes:
- requires:
- - build
- - start_upgrade_tests:
- type: approval
- - j8_upgradetests-no-vnodes:
- requires:
- - start_upgrade_tests
- - build
- - start_j8_repeated_utest:
- type: approval
- - j8_repeated_utest:
+ - j8_dtests:
requires:
- - start_j8_repeated_utest
- build
- - start_j8_repeated_dtest:
- type: approval
- - j8_repeated_dtest:
+ - j8_dtests_vnode:
requires:
- - start_j8_repeated_dtest
- build
- - start_repeated_upgrade_dtest:
+ - start_upgrade_dtests:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_repeated_upgrade_dtest
+ - start_upgrade_dtests
- build
- - start_repeated_jvm_upgrade_dtest:
- type: approval
- - repeated_jvm_upgrade_dtest:
- requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
diff --git a/.circleci/config.yml.HIGHRES b/.circleci/config.yml.HIGHRES
index 7a93ab4b3d..b4983f9b8a 100644
--- a/.circleci/config.yml.HIGHRES
+++ b/.circleci/config.yml.HIGHRES
@@ -108,25 +108,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_jvm_upgrade_dtest:
+ utests_compression_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -159,81 +160,80 @@ jobs:
which java
java -version
- run:
- name: Run repeated JUnit test
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_JVM_UPGRADE_DTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_JVM_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running test-jvm-dtest-some ${REPEATED_JVM_UPGRADE_DTEST_CLASS} ${REPEATED_JVM_UPGRADE_DTEST_METHODS} ${REPEATED_JVM_UPGRADE_DTEST_COUNT} times"
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- target=test-jvm-dtest-some
- class_path=${REPEATED_JVM_UPGRADE_DTEST_CLASS}
- class_name="${class_path##*.}"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
# Prepare the -Dtest.name argument.
# It can be the fully qualified class name or the short class name, depending on the target.
if [[ $target == "test" || \
$target == "test-cdc" || \
$target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
else
- name="-Dtest.name=$class_path"
+ name_arg="-Dtest.name=$class"
fi
# Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_JVM_UPGRADE_DTEST_METHODS}" == "<nil>" ]; then
- methods=""
+ if [ $method == "" ]; then
+ methods_arg=""
else
- methods="-Dtest.methods=${REPEATED_JVM_UPGRADE_DTEST_METHODS}"
+ methods_arg="-Dtest.methods=$method"
fi
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
+ echo "Running test $test, iteration $i of $count"
# run the test
status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ if !( set -o pipefail && \
+ ant test-compression $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
status="fails"
exit_code=1
fi
# move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
mkdir -p $dest
- mv stdout.txt $dest/test-jvm-dtest-some-${REPEATED_JVM_UPGRADE_DTEST_CLASS}.txt
+ mv stdout.txt $dest/${test}.txt
# move the XML output files
source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
@@ -241,31 +241,29 @@ jobs:
# move the log files
source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
fi
# maybe stop iterations on test failure
- if [[ ${REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
break
fi
done
-
- (exit ${exit_code})
- fi
- fi
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/results/repeated_utest/stdout
+ path: /tmp/results/repeated_utests/stdout
destination: stdout
- store_artifacts:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
destination: junitxml
- store_artifacts:
- path: /tmp/results/repeated_utest/logs
+ path: /tmp/results/repeated_utests/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -282,25 +280,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_upgrade_dtest:
+ utests_long_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -311,82 +310,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_UPGRADE_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UPGRADE_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_LONG_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_LONG_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS_LONG} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if false; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if true; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_UPGRADE_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant long-testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -402,25 +452,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_dtest:
+ j8_unit_tests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -431,77 +482,251 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if ${REPEATED_DTEST_VNODES}; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if false; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
- - store_test_results:
- path: /tmp/results
- - store_artifacts:
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_UPGRADE_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_UPGRADE_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_UPGRADE_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if true; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
path: /tmp/dtest
destination: dtest
- store_artifacts:
@@ -522,22 +747,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
build:
@@ -619,25 +845,621 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_dtests_vnode_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if true; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Determine Tests to Run (j8_upgradetests_without_vnodes)
+ no_output_timeout: 5m
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ cd cassandra-dtest
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
+ set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
+ if [ -z '^upgrade_tests' ]; then
+ mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
+ else
+ grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
+ fi
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ - run:
+ name: Run dtests (j8_upgradetests_without_vnodes)
+ no_output_timeout: 15m
+ command: |
+ echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
+ set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest_j8_upgradetests_without_vnodes
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_j8_upgradetests_without_vnodes_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_jvm_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_UPGRADE_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_unit_tests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Determine unit Tests to Run
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ rm -fr ~/cassandra-dtest/upgrade_tests
+ echo "***java tests***"
+
+ # get all of our unit test filenames
+ set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
+
+ # split up the unit tests into groups based on the number of containers we have
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
+ set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
+ cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ no_output_timeout: 15m
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Run Unit Tests (testclasslist)
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
+ if [ -z "$test_timeout" ]; then
+ test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ fi
+ ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
+ no_output_timeout: 15m
+ - store_test_results:
+ path: /tmp/cassandra/build/test/output/
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_dtests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 100
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Determine Tests to Run (j8_without_vnodes)
+ no_output_timeout: 5m
+ command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_without_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --skip-resource-in [...]
+ - run:
+ name: Run dtests (j8_without_vnodes)
+ no_output_timeout: 15m
+ command: "echo \"cat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not [...]
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest_j8_without_vnodes
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_j8_without_vnodes_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 2048M
+ - CCM_HEAP_NEWSIZE: 512M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-no-vnodes:
+ j8_dtests_vnode:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -663,21 +1485,21 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_without_vnodes)
+ name: Determine Tests to Run (j8_with_vnodes)
no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_without_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --skip-resource-in [...]
+ command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- run:
- name: Run dtests (j8_without_vnodes)
+ name: Run dtests (j8_with_vnodes)
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not [...]
+ command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_without_vnodes
+ destination: dtest_j8_with_vnodes
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_without_vnodes_logs
+ destination: dtest_j8_with_vnodes_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -693,25 +1515,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_upgradetests-no-vnodes:
+ j8_jvm_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -722,77 +1545,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Determine Tests to Run (j8_upgradetests_without_vnodes)
- no_output_timeout: 5m
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- cd cassandra-dtest
- source ~/env3.6/bin/activate
+ set -x
export PATH=$JAVA_HOME/bin:$PATH
-
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
fi
- echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
- set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
- if [ -z '^upgrade_tests' ]; then
- mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
- else
- grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
fi
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- - run:
- name: Run dtests (j8_upgradetests_without_vnodes)
- no_output_timeout: 15m
- command: |
- echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
- fi
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
- set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest_j8_upgradetests_without_vnodes
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_j8_upgradetests_without_vnodes_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -808,25 +1687,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_unit_tests:
+ j8_repeated_ant_test:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -836,25 +1716,6 @@ jobs:
steps:
- attach_workspace:
at: /home/cassandra
- - run:
- name: Determine unit Tests to Run
- command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- rm -fr ~/cassandra-dtest/upgrade_tests
- echo "***java tests***"
-
- # get all of our unit test filenames
- set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
-
- # split up the unit tests into groups based on the number of containers we have
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
- set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
- cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- no_output_timeout: 15m
- run:
name: Log Environment Information
command: |
@@ -878,28 +1739,114 @@ jobs:
which java
java -version
- run:
- name: Run Unit Tests (testclasslist)
+ name: Run repeated JUnit test
+ no_output_timeout: 15m
command: |
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
- test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
- if [ -z "$test_timeout" ]; then
- test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ if [ "${REPEATED_ANT_TEST_CLASS}" == "<nil>" ]; then
+ echo "Repeated utest class name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" == "<nil>" ]; then
+ echo "Repeated utest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" -le 0 ]; then
+ echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_ANT_TEST_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_ANT_TEST_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_ANT_TEST_TARGET} ${REPEATED_ANT_TEST_CLASS} ${REPEATED_ANT_TEST_METHODS} ${REPEATED_ANT_TEST_COUNT} times"
+
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ target=${REPEATED_ANT_TEST_TARGET}
+ class_path=${REPEATED_ANT_TEST_CLASS}
+ class_name="${class_path##*.}"
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name="-Dtest.name=$class_name"
+ else
+ name="-Dtest.name=$class_path"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ "${REPEATED_ANT_TEST_METHODS}" == "<nil>" ]; then
+ methods=""
+ else
+ methods="-Dtest.methods=${REPEATED_ANT_TEST_METHODS}"
+ fi
+
+ # Run the test target as many times as requested collecting the exit code,
+ # stopping the iteration only if stop_on_failure is set.
+ exit_code="$?"
+ for i in $(seq -w 1 $count); do
+
+ echo "Running test iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${REPEATED_ANT_TEST_TARGET}-${REPEATED_ANT_TEST_CLASS}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utest/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+
+ (exit ${exit_code})
+ fi
fi
- ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
- no_output_timeout: 15m
- store_test_results:
- path: /tmp/cassandra/build/test/output/
+ path: /tmp/results/repeated_utest/output
- store_artifacts:
- path: /tmp/cassandra/build/test/output
+ path: /tmp/results/repeated_utest/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utest/output
destination: junitxml
- store_artifacts:
- path: /tmp/cassandra/build/test/logs
+ path: /tmp/results/repeated_utest/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -916,25 +1863,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-with-vnodes:
+ j8_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -960,21 +1908,69 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_with_vnodes)
- no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- - run:
- name: Run dtests (j8_with_vnodes)
+ name: Run repeated Python dtest
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_with_vnodes
+ destination: dtest
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_with_vnodes_logs
+ destination: dtest_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -990,22 +1986,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_jvm_dtests:
@@ -1098,22 +2095,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_long:
@@ -1160,22 +2158,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_compression:
@@ -1268,196 +2267,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_utest:
- docker:
- - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
- resource_class: xlarge
- working_directory: ~/
- shell: /bin/bash -eo pipefail -l
- parallelism: 100
- steps:
- - attach_workspace:
- at: /home/cassandra
- - run:
- name: Log Environment Information
- command: |
- echo '*** id ***'
- id
- echo '*** cat /proc/cpuinfo ***'
- cat /proc/cpuinfo
- echo '*** free -m ***'
- free -m
- echo '*** df -m ***'
- df -m
- echo '*** ifconfig -a ***'
- ifconfig -a
- echo '*** uname -a ***'
- uname -a
- echo '*** mount ***'
- mount
- echo '*** env ***'
- env
- echo '*** java ***'
- which java
- java -version
- - run:
- name: Run repeated JUnit test
- no_output_timeout: 15m
- command: |
- if [ "${REPEATED_UTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UTEST_TARGET} ${REPEATED_UTEST_CLASS} ${REPEATED_UTEST_METHODS} ${REPEATED_UTEST_COUNT} times"
-
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
-
- target=${REPEATED_UTEST_TARGET}
- class_path=${REPEATED_UTEST_CLASS}
- class_name="${class_path##*.}"
-
- # Prepare the -Dtest.name argument.
- # It can be the fully qualified class name or the short class name, depending on the target.
- if [[ $target == "test" || \
- $target == "test-cdc" || \
- $target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
- else
- name="-Dtest.name=$class_path"
- fi
-
- # Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_UTEST_METHODS}" == "<nil>" ]; then
- methods=""
- else
- methods="-Dtest.methods=${REPEATED_UTEST_METHODS}"
- fi
-
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
- for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
-
- # run the test
- status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
- status="fails"
- exit_code=1
- fi
-
- # move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
- mkdir -p $dest
- mv stdout.txt $dest/${REPEATED_UTEST_TARGET}-${REPEATED_UTEST_CLASS}.txt
-
- # move the XML output files
- source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # move the log files
- source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # maybe stop iterations on test failure
- if [[ ${REPEATED_UTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
- break
- fi
- done
-
- (exit ${exit_code})
- fi
- fi
- - store_test_results:
- path: /tmp/results/repeated_utest/output
- - store_artifacts:
- path: /tmp/results/repeated_utest/stdout
- destination: stdout
- - store_artifacts:
- path: /tmp/results/repeated_utest/output
- destination: junitxml
- - store_artifacts:
- path: /tmp/results/repeated_utest/logs
- destination: logs
- environment:
- - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - ANT_HOME: /usr/share/ant
- - LANG: en_US.UTF-8
- - KEEP_TEST_DIR: true
- - DEFAULT_DIR: /home/cassandra/cassandra-dtest
- - PYTHONIOENCODING: utf-8
- - PYTHONUNBUFFERED: true
- - CASS_DRIVER_NO_EXTENSIONS: true
- - CASS_DRIVER_NO_CYTHON: true
- - CASSANDRA_SKIP_SYNC: true
- - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: trunk
- - CCM_MAX_HEAP_SIZE: 2048M
- - CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_dtest_jars_build:
@@ -1536,22 +2362,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 2048M
- CCM_HEAP_NEWSIZE: 512M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
workflows:
@@ -1569,76 +2396,108 @@ workflows:
requires:
- start_j8_unit_tests
- build
+ - start_j8_unit_tests_repeat:
+ type: approval
+ - j8_unit_tests_repeat:
+ requires:
+ - start_j8_unit_tests_repeat
+ - build
- start_j8_jvm_dtests:
type: approval
- j8_jvm_dtests:
requires:
- start_j8_jvm_dtests
- build
+ - start_j8_jvm_dtests_repeat:
+ type: approval
+ - j8_jvm_dtests_repeat:
+ requires:
+ - start_j8_jvm_dtests_repeat
+ - build
- start_utests_long:
type: approval
- utests_long:
requires:
- start_utests_long
- build
+ - start_utests_long_repeat:
+ type: approval
+ - utests_long_repeat:
+ requires:
+ - start_utests_long_repeat
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
+ - start_utests_compression_repeat:
+ type: approval
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression_repeat
+ - build
- start_j8_dtest_jars_build:
type: approval
- j8_dtest_jars_build:
requires:
- start_j8_dtest_jars_build
- build
- - start_jvm_upgrade_dtest:
+ - start_jvm_upgrade_dtests:
type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
+ - start_jvm_upgrade_dtests
+ - j8_dtest_jars_build
+ - start_jvm_upgrade_dtests_repeat:
+ type: approval
+ - j8_jvm_upgrade_dtests_repeat:
+ requires:
+ - start_jvm_upgrade_dtests_repeat
- j8_dtest_jars_build
- start_j8_dtests:
type: approval
- - j8_dtests-with-vnodes:
+ - j8_dtests:
requires:
- start_j8_dtests
- build
- - j8_dtests-no-vnodes:
+ - start_j8_dtests_repeat:
+ type: approval
+ - j8_dtests_repeat:
requires:
- - start_j8_dtests
+ - start_j8_dtests_repeat
- build
- - start_upgrade_tests:
+ - start_j8_dtests_vnode:
type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_vnode:
requires:
- - start_upgrade_tests
+ - start_j8_dtests_vnode
- build
- - start_j8_repeated_utest:
+ - start_j8_dtests_vnode_repeat:
type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_utest
+ - start_j8_dtests_vnode_repeat
- build
- - start_j8_repeated_dtest:
+ - start_j8_upgrade_dtests:
type: approval
- - j8_repeated_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_j8_repeated_dtest
+ - start_j8_upgrade_dtests
- build
- - start_repeated_upgrade_dtest:
+ - start_j8_upgrade_dtests_repeat:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_upgrade_dtest
+ - start_j8_upgrade_dtests_repeat
- build
- - start_repeated_jvm_upgrade_dtest:
+ - start_j8_repeated_ant_test:
type: approval
- - repeated_jvm_upgrade_dtest:
+ - j8_repeated_ant_test:
requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
+ - start_j8_repeated_ant_test
+ - build
pre-commit_tests:
jobs:
- start_pre-commit_tests:
@@ -1649,66 +2508,66 @@ workflows:
- j8_unit_tests:
requires:
- build
+ - j8_unit_tests_repeat:
+ requires:
+ - build
- j8_jvm_dtests:
requires:
- build
+ - j8_jvm_dtests_repeat:
+ requires:
+ - build
- start_utests_long:
type: approval
- utests_long:
requires:
- start_utests_long
- build
+ - utests_long_repeat:
+ requires:
+ - start_utests_long
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
- - start_j8_dtest_jars_build:
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression
+ - build
+ - start_jvm_upgrade_dtests:
type: approval
- j8_dtest_jars_build:
requires:
- - start_j8_dtest_jars_build
+ - start_jvm_upgrade_dtests
- build
- - start_jvm_upgrade_dtest:
- type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
- j8_dtest_jars_build
- - j8_dtests-with-vnodes:
+ - j8_jvm_upgrade_dtests_repeat:
requires:
- - build
- - j8_dtests-no-vnodes:
+ - j8_dtest_jars_build
+ - j8_dtests:
requires:
- build
- - start_upgrade_tests:
- type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_repeat:
requires:
- - start_upgrade_tests
- build
- - start_j8_repeated_utest:
- type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode:
requires:
- - start_j8_repeated_utest
- build
- - start_j8_repeated_dtest:
- type: approval
- - j8_repeated_dtest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_dtest
- build
- - start_repeated_upgrade_dtest:
+ - start_upgrade_dtests:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_repeated_upgrade_dtest
+ - start_upgrade_dtests
- build
- - start_repeated_jvm_upgrade_dtest:
- type: approval
- - repeated_jvm_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
+ - start_upgrade_dtests
+ - build
diff --git a/.circleci/config.yml.LOWRES b/.circleci/config.yml.LOWRES
index a3fd9e1a2d..baeb67190a 100644
--- a/.circleci/config.yml.LOWRES
+++ b/.circleci/config.yml.LOWRES
@@ -108,25 +108,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_jvm_upgrade_dtest:
+ utests_compression_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -159,81 +160,80 @@ jobs:
which java
java -version
- run:
- name: Run repeated JUnit test
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_JVM_UPGRADE_DTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_JVM_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running test-jvm-dtest-some ${REPEATED_JVM_UPGRADE_DTEST_CLASS} ${REPEATED_JVM_UPGRADE_DTEST_METHODS} ${REPEATED_JVM_UPGRADE_DTEST_COUNT} times"
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- target=test-jvm-dtest-some
- class_path=${REPEATED_JVM_UPGRADE_DTEST_CLASS}
- class_name="${class_path##*.}"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
# Prepare the -Dtest.name argument.
# It can be the fully qualified class name or the short class name, depending on the target.
if [[ $target == "test" || \
$target == "test-cdc" || \
$target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
else
- name="-Dtest.name=$class_path"
+ name_arg="-Dtest.name=$class"
fi
# Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_JVM_UPGRADE_DTEST_METHODS}" == "<nil>" ]; then
- methods=""
+ if [ $method == "" ]; then
+ methods_arg=""
else
- methods="-Dtest.methods=${REPEATED_JVM_UPGRADE_DTEST_METHODS}"
+ methods_arg="-Dtest.methods=$method"
fi
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
+ echo "Running test $test, iteration $i of $count"
# run the test
status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ if !( set -o pipefail && \
+ ant test-compression $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
status="fails"
exit_code=1
fi
# move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
mkdir -p $dest
- mv stdout.txt $dest/test-jvm-dtest-some-${REPEATED_JVM_UPGRADE_DTEST_CLASS}.txt
+ mv stdout.txt $dest/${test}.txt
# move the XML output files
source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
@@ -241,31 +241,29 @@ jobs:
# move the log files
source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
fi
# maybe stop iterations on test failure
- if [[ ${REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
break
fi
done
-
- (exit ${exit_code})
- fi
- fi
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/results/repeated_utest/stdout
+ path: /tmp/results/repeated_utests/stdout
destination: stdout
- store_artifacts:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
destination: junitxml
- store_artifacts:
- path: /tmp/results/repeated_utest/logs
+ path: /tmp/results/repeated_utests/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -282,25 +280,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_upgrade_dtest:
+ utests_long_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -311,82 +310,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_UPGRADE_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UPGRADE_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_LONG_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_LONG_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS_LONG} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if false; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if true; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_UPGRADE_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant long-testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -402,25 +452,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_dtest:
+ j8_unit_tests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -431,77 +482,251 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if ${REPEATED_DTEST_VNODES}; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if false; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
- - store_test_results:
- path: /tmp/results
- - store_artifacts:
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_UPGRADE_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_UPGRADE_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_UPGRADE_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if true; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
path: /tmp/dtest
destination: dtest
- store_artifacts:
@@ -522,22 +747,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
build:
@@ -619,25 +845,621 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_dtests_vnode_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if true; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Determine Tests to Run (j8_upgradetests_without_vnodes)
+ no_output_timeout: 5m
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ cd cassandra-dtest
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
+ set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
+ if [ -z '^upgrade_tests' ]; then
+ mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
+ else
+ grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
+ fi
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+ - run:
+ name: Run dtests (j8_upgradetests_without_vnodes)
+ no_output_timeout: 15m
+ command: |
+ echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
+ cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
+ export RUN_STATIC_UPGRADE_MATRIX=true
+ fi
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
+ set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest_j8_upgradetests_without_vnodes
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_j8_upgradetests_without_vnodes_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_jvm_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_UPGRADE_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_unit_tests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Determine unit Tests to Run
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ rm -fr ~/cassandra-dtest/upgrade_tests
+ echo "***java tests***"
+
+ # get all of our unit test filenames
+ set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
+
+ # split up the unit tests into groups based on the number of containers we have
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
+ set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
+ cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ no_output_timeout: 15m
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Run Unit Tests (testclasslist)
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
+ if [ -z "$test_timeout" ]; then
+ test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ fi
+ ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
+ no_output_timeout: 15m
+ - store_test_results:
+ path: /tmp/cassandra/build/test/output/
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/cassandra/build/test/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_dtests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 4
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Determine Tests to Run (j8_without_vnodes)
+ no_output_timeout: 5m
+ command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_without_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --skip-resource-in [...]
+ - run:
+ name: Run dtests (j8_without_vnodes)
+ no_output_timeout: 15m
+ command: "echo \"cat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not [...]
+ - store_test_results:
+ path: /tmp/results
+ - store_artifacts:
+ path: /tmp/dtest
+ destination: dtest_j8_without_vnodes
+ - store_artifacts:
+ path: ~/cassandra-dtest/logs
+ destination: dtest_j8_without_vnodes_logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-no-vnodes:
+ j8_dtests_vnode:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -663,21 +1485,21 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_without_vnodes)
+ name: Determine Tests to Run (j8_with_vnodes)
no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_without_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --skip-resource-in [...]
+ command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- run:
- name: Run dtests (j8_without_vnodes)
+ name: Run dtests (j8_with_vnodes)
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not [...]
+ command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_without_vnodes
+ destination: dtest_j8_with_vnodes
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_without_vnodes_logs
+ destination: dtest_j8_with_vnodes_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -693,25 +1515,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_upgradetests-no-vnodes:
+ j8_jvm_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -722,77 +1545,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Determine Tests to Run (j8_upgradetests_without_vnodes)
- no_output_timeout: 5m
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- cd cassandra-dtest
- source ~/env3.6/bin/activate
+ set -x
export PATH=$JAVA_HOME/bin:$PATH
-
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
fi
- echo "***Collected DTests (j8_upgradetests_without_vnodes)***"
- set -eo pipefail && ./run_dtests.py --execute-upgrade-tests --dtest-print-tests-only --dtest-print-tests-output=/tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw --cassandra-dir=../cassandra
- if [ -z '^upgrade_tests' ]; then
- mv /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw /tmp/all_dtest_tests_j8_upgradetests_without_vnodes
- else
- grep -e '^upgrade_tests' /tmp/all_dtest_tests_j8_upgradetests_without_vnodes_raw > /tmp/all_dtest_tests_j8_upgradetests_without_vnodes || { echo "Filter did not match any tests! Exiting build."; exit 0; }
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
fi
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=classname /tmp/all_dtest_tests_j8_upgradetests_without_vnodes > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes.txt | tr '\n' ' ' > /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- - run:
- name: Run dtests (j8_upgradetests_without_vnodes)
- no_output_timeout: 15m
- command: |
- echo "cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt"
- cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- if [ -n 'RUN_STATIC_UPGRADE_MATRIX=true' ]; then
- export RUN_STATIC_UPGRADE_MATRIX=true
- fi
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- export SPLIT_TESTS=`cat /tmp/split_dtest_tests_j8_upgradetests_without_vnodes_final.txt`
- set -o pipefail && cd ~/cassandra-dtest && pytest --execute-upgrade-tests --log-level="INFO" --junit-xml=/tmp/results/dtests/pytest_result_j8_upgradetests_without_vnodes.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $SPLIT_TESTS 2>&1 | tee /tmp/dtest/stdout.txt
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest_j8_upgradetests_without_vnodes
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_j8_upgradetests_without_vnodes_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -808,25 +1687,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_unit_tests:
+ j8_repeated_ant_test:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -836,25 +1716,6 @@ jobs:
steps:
- attach_workspace:
at: /home/cassandra
- - run:
- name: Determine unit Tests to Run
- command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- rm -fr ~/cassandra-dtest/upgrade_tests
- echo "***java tests***"
-
- # get all of our unit test filenames
- set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
-
- # split up the unit tests into groups based on the number of containers we have
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
- set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
- cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- no_output_timeout: 15m
- run:
name: Log Environment Information
command: |
@@ -878,28 +1739,114 @@ jobs:
which java
java -version
- run:
- name: Run Unit Tests (testclasslist)
+ name: Run repeated JUnit test
+ no_output_timeout: 15m
command: |
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
- test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
- if [ -z "$test_timeout" ]; then
- test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ if [ "${REPEATED_ANT_TEST_CLASS}" == "<nil>" ]; then
+ echo "Repeated utest class name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" == "<nil>" ]; then
+ echo "Repeated utest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_ANT_TEST_COUNT}" -le 0 ]; then
+ echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_ANT_TEST_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_ANT_TEST_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_ANT_TEST_TARGET} ${REPEATED_ANT_TEST_CLASS} ${REPEATED_ANT_TEST_METHODS} ${REPEATED_ANT_TEST_COUNT} times"
+
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+
+ target=${REPEATED_ANT_TEST_TARGET}
+ class_path=${REPEATED_ANT_TEST_CLASS}
+ class_name="${class_path##*.}"
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name="-Dtest.name=$class_name"
+ else
+ name="-Dtest.name=$class_path"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ "${REPEATED_ANT_TEST_METHODS}" == "<nil>" ]; then
+ methods=""
+ else
+ methods="-Dtest.methods=${REPEATED_ANT_TEST_METHODS}"
+ fi
+
+ # Run the test target as many times as requested collecting the exit code,
+ # stopping the iteration only if stop_on_failure is set.
+ exit_code="$?"
+ for i in $(seq -w 1 $count); do
+
+ echo "Running test iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${REPEATED_ANT_TEST_TARGET}-${REPEATED_ANT_TEST_CLASS}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utest/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+
+ (exit ${exit_code})
+ fi
fi
- ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
- no_output_timeout: 15m
- store_test_results:
- path: /tmp/cassandra/build/test/output/
+ path: /tmp/results/repeated_utest/output
- store_artifacts:
- path: /tmp/cassandra/build/test/output
+ path: /tmp/results/repeated_utest/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utest/output
destination: junitxml
- store_artifacts:
- path: /tmp/cassandra/build/test/logs
+ path: /tmp/results/repeated_utest/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -916,25 +1863,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-with-vnodes:
+ j8_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -960,21 +1908,69 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_with_vnodes)
- no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_with_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --use-vnodes --skip-r [...]
- - run:
- name: Run dtests (j8_with_vnodes)
+ name: Run repeated Python dtest
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_with_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not the ex [...]
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_with_vnodes
+ destination: dtest
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_with_vnodes_logs
+ destination: dtest_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -990,22 +1986,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_jvm_dtests:
@@ -1098,22 +2095,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_long:
@@ -1160,22 +2158,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
utests_compression:
@@ -1268,196 +2267,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_utest:
- docker:
- - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
- resource_class: medium
- working_directory: ~/
- shell: /bin/bash -eo pipefail -l
- parallelism: 4
- steps:
- - attach_workspace:
- at: /home/cassandra
- - run:
- name: Log Environment Information
- command: |
- echo '*** id ***'
- id
- echo '*** cat /proc/cpuinfo ***'
- cat /proc/cpuinfo
- echo '*** free -m ***'
- free -m
- echo '*** df -m ***'
- df -m
- echo '*** ifconfig -a ***'
- ifconfig -a
- echo '*** uname -a ***'
- uname -a
- echo '*** mount ***'
- mount
- echo '*** env ***'
- env
- echo '*** java ***'
- which java
- java -version
- - run:
- name: Run repeated JUnit test
- no_output_timeout: 15m
- command: |
- if [ "${REPEATED_UTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UTEST_TARGET} ${REPEATED_UTEST_CLASS} ${REPEATED_UTEST_METHODS} ${REPEATED_UTEST_COUNT} times"
-
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
-
- target=${REPEATED_UTEST_TARGET}
- class_path=${REPEATED_UTEST_CLASS}
- class_name="${class_path##*.}"
-
- # Prepare the -Dtest.name argument.
- # It can be the fully qualified class name or the short class name, depending on the target.
- if [[ $target == "test" || \
- $target == "test-cdc" || \
- $target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
- else
- name="-Dtest.name=$class_path"
- fi
-
- # Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_UTEST_METHODS}" == "<nil>" ]; then
- methods=""
- else
- methods="-Dtest.methods=${REPEATED_UTEST_METHODS}"
- fi
-
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
- for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
-
- # run the test
- status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
- status="fails"
- exit_code=1
- fi
-
- # move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
- mkdir -p $dest
- mv stdout.txt $dest/${REPEATED_UTEST_TARGET}-${REPEATED_UTEST_CLASS}.txt
-
- # move the XML output files
- source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # move the log files
- source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
- mkdir -p $dest
- if [[ -d $source && -n "$(ls $source)" ]]; then
- mv $source/* $dest/
- fi
-
- # maybe stop iterations on test failure
- if [[ ${REPEATED_UTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
- break
- fi
- done
-
- (exit ${exit_code})
- fi
- fi
- - store_test_results:
- path: /tmp/results/repeated_utest/output
- - store_artifacts:
- path: /tmp/results/repeated_utest/stdout
- destination: stdout
- - store_artifacts:
- path: /tmp/results/repeated_utest/output
- destination: junitxml
- - store_artifacts:
- path: /tmp/results/repeated_utest/logs
- destination: logs
- environment:
- - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- - ANT_HOME: /usr/share/ant
- - LANG: en_US.UTF-8
- - KEEP_TEST_DIR: true
- - DEFAULT_DIR: /home/cassandra/cassandra-dtest
- - PYTHONIOENCODING: utf-8
- - PYTHONUNBUFFERED: true
- - CASS_DRIVER_NO_EXTENSIONS: true
- - CASS_DRIVER_NO_CYTHON: true
- - CASSANDRA_SKIP_SYNC: true
- - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
- - DTEST_BRANCH: trunk
- - CCM_MAX_HEAP_SIZE: 1024M
- - CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
j8_dtest_jars_build:
@@ -1536,22 +2362,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
workflows:
@@ -1569,76 +2396,108 @@ workflows:
requires:
- start_j8_unit_tests
- build
+ - start_j8_unit_tests_repeat:
+ type: approval
+ - j8_unit_tests_repeat:
+ requires:
+ - start_j8_unit_tests_repeat
+ - build
- start_j8_jvm_dtests:
type: approval
- j8_jvm_dtests:
requires:
- start_j8_jvm_dtests
- build
+ - start_j8_jvm_dtests_repeat:
+ type: approval
+ - j8_jvm_dtests_repeat:
+ requires:
+ - start_j8_jvm_dtests_repeat
+ - build
- start_utests_long:
type: approval
- utests_long:
requires:
- start_utests_long
- build
+ - start_utests_long_repeat:
+ type: approval
+ - utests_long_repeat:
+ requires:
+ - start_utests_long_repeat
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
+ - start_utests_compression_repeat:
+ type: approval
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression_repeat
+ - build
- start_j8_dtest_jars_build:
type: approval
- j8_dtest_jars_build:
requires:
- start_j8_dtest_jars_build
- build
- - start_jvm_upgrade_dtest:
+ - start_jvm_upgrade_dtests:
type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
+ - start_jvm_upgrade_dtests
+ - j8_dtest_jars_build
+ - start_jvm_upgrade_dtests_repeat:
+ type: approval
+ - j8_jvm_upgrade_dtests_repeat:
+ requires:
+ - start_jvm_upgrade_dtests_repeat
- j8_dtest_jars_build
- start_j8_dtests:
type: approval
- - j8_dtests-with-vnodes:
+ - j8_dtests:
requires:
- start_j8_dtests
- build
- - j8_dtests-no-vnodes:
+ - start_j8_dtests_repeat:
+ type: approval
+ - j8_dtests_repeat:
requires:
- - start_j8_dtests
+ - start_j8_dtests_repeat
- build
- - start_upgrade_tests:
+ - start_j8_dtests_vnode:
type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_vnode:
requires:
- - start_upgrade_tests
+ - start_j8_dtests_vnode
- build
- - start_j8_repeated_utest:
+ - start_j8_dtests_vnode_repeat:
type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_utest
+ - start_j8_dtests_vnode_repeat
- build
- - start_j8_repeated_dtest:
+ - start_j8_upgrade_dtests:
type: approval
- - j8_repeated_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_j8_repeated_dtest
+ - start_j8_upgrade_dtests
- build
- - start_repeated_upgrade_dtest:
+ - start_j8_upgrade_dtests_repeat:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_upgrade_dtest
+ - start_j8_upgrade_dtests_repeat
- build
- - start_repeated_jvm_upgrade_dtest:
+ - start_j8_repeated_ant_test:
type: approval
- - repeated_jvm_upgrade_dtest:
+ - j8_repeated_ant_test:
requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
+ - start_j8_repeated_ant_test
+ - build
pre-commit_tests:
jobs:
- start_pre-commit_tests:
@@ -1649,66 +2508,66 @@ workflows:
- j8_unit_tests:
requires:
- build
+ - j8_unit_tests_repeat:
+ requires:
+ - build
- j8_jvm_dtests:
requires:
- build
+ - j8_jvm_dtests_repeat:
+ requires:
+ - build
- start_utests_long:
type: approval
- utests_long:
requires:
- start_utests_long
- build
+ - utests_long_repeat:
+ requires:
+ - start_utests_long
+ - build
- start_utests_compression:
type: approval
- utests_compression:
requires:
- start_utests_compression
- build
- - start_j8_dtest_jars_build:
+ - utests_compression_repeat:
+ requires:
+ - start_utests_compression
+ - build
+ - start_jvm_upgrade_dtests:
type: approval
- j8_dtest_jars_build:
requires:
- - start_j8_dtest_jars_build
+ - start_jvm_upgrade_dtests
- build
- - start_jvm_upgrade_dtest:
- type: approval
- j8_jvm_upgrade_dtests:
requires:
- - start_jvm_upgrade_dtest
- j8_dtest_jars_build
- - j8_dtests-with-vnodes:
+ - j8_jvm_upgrade_dtests_repeat:
requires:
- - build
- - j8_dtests-no-vnodes:
+ - j8_dtest_jars_build
+ - j8_dtests:
requires:
- build
- - start_upgrade_tests:
- type: approval
- - j8_upgradetests-no-vnodes:
+ - j8_dtests_repeat:
requires:
- - start_upgrade_tests
- build
- - start_j8_repeated_utest:
- type: approval
- - j8_repeated_utest:
+ - j8_dtests_vnode:
requires:
- - start_j8_repeated_utest
- build
- - start_j8_repeated_dtest:
- type: approval
- - j8_repeated_dtest:
+ - j8_dtests_vnode_repeat:
requires:
- - start_j8_repeated_dtest
- build
- - start_repeated_upgrade_dtest:
+ - start_upgrade_dtests:
type: approval
- - repeated_upgrade_dtest:
+ - j8_upgrade_dtests:
requires:
- - start_repeated_upgrade_dtest
+ - start_upgrade_dtests
- build
- - start_repeated_jvm_upgrade_dtest:
- type: approval
- - repeated_jvm_upgrade_dtest:
+ - j8_upgrade_dtests_repeat:
requires:
- - start_repeated_jvm_upgrade_dtest
- - j8_dtest_jars_build
+ - start_upgrade_dtests
+ - build
diff --git a/.circleci/config.yml.MIDRES b/.circleci/config.yml.MIDRES
index d13d6643f9..38a2fa39ef 100644
--- a/.circleci/config.yml.MIDRES
+++ b/.circleci/config.yml.MIDRES
@@ -108,25 +108,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_jvm_upgrade_dtest:
+ utests_compression_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -159,81 +160,80 @@ jobs:
which java
java -version
- run:
- name: Run repeated JUnit test
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_JVM_UPGRADE_DTEST_CLASS}" == "<nil>" ]; then
- echo "Repeated utest class name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated utest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_JVM_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated utest count is lesser or equals than zero, exiting without running any test"
- else
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_JVM_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running test-jvm-dtest-some ${REPEATED_JVM_UPGRADE_DTEST_CLASS} ${REPEATED_JVM_UPGRADE_DTEST_METHODS} ${REPEATED_JVM_UPGRADE_DTEST_COUNT} times"
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- set -x
- export PATH=$JAVA_HOME/bin:$PATH
- time mv ~/cassandra /tmp
- cd /tmp/cassandra
- if [ -d ~/dtest_jars ]; then
- cp ~/dtest_jars/dtest* /tmp/cassandra/build/
- fi
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- target=test-jvm-dtest-some
- class_path=${REPEATED_JVM_UPGRADE_DTEST_CLASS}
- class_name="${class_path##*.}"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
# Prepare the -Dtest.name argument.
# It can be the fully qualified class name or the short class name, depending on the target.
if [[ $target == "test" || \
$target == "test-cdc" || \
$target == "test-compression" || \
- $target == "test-system-keyspace-directory" ]]; then
- name="-Dtest.name=$class_name"
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
else
- name="-Dtest.name=$class_path"
+ name_arg="-Dtest.name=$class"
fi
# Prepare the -Dtest.methods argument, which is optional
- if [ "${REPEATED_JVM_UPGRADE_DTEST_METHODS}" == "<nil>" ]; then
- methods=""
+ if [ $method == "" ]; then
+ methods_arg=""
else
- methods="-Dtest.methods=${REPEATED_JVM_UPGRADE_DTEST_METHODS}"
+ methods_arg="-Dtest.methods=$method"
fi
- # Run the test target as many times as requested collecting the exit code,
- # stopping the iteration only if stop_on_failure is set.
- exit_code="$?"
for i in $(seq -w 1 $count); do
-
- echo "Running test iteration $i of $count"
+ echo "Running test $test, iteration $i of $count"
# run the test
status="passes"
- if !( set -o pipefail && ant $target $name $methods -Dno-build-test=true | tee stdout.txt ); then
+ if !( set -o pipefail && \
+ ant test-compression $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
status="fails"
exit_code=1
fi
# move the stdout output file
- dest=/tmp/results/repeated_utest/stdout/${status}/${i}
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
mkdir -p $dest
- mv stdout.txt $dest/test-jvm-dtest-some-${REPEATED_JVM_UPGRADE_DTEST_CLASS}.txt
+ mv stdout.txt $dest/${test}.txt
# move the XML output files
source=build/test/output
- dest=/tmp/results/repeated_utest/output/${status}/${i}
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
@@ -241,31 +241,29 @@ jobs:
# move the log files
source=build/test/logs
- dest=/tmp/results/repeated_utest/logs/${status}/${i}
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
mkdir -p $dest
if [[ -d $source && -n "$(ls $source)" ]]; then
mv $source/* $dest/
fi
# maybe stop iterations on test failure
- if [[ ${REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
break
fi
done
-
- (exit ${exit_code})
- fi
- fi
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/results/repeated_utest/stdout
+ path: /tmp/results/repeated_utests/stdout
destination: stdout
- store_artifacts:
- path: /tmp/results/repeated_utest/output
+ path: /tmp/results/repeated_utests/output
destination: junitxml
- store_artifacts:
- path: /tmp/results/repeated_utest/logs
+ path: /tmp/results/repeated_utests/logs
destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
@@ -282,28 +280,29 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- repeated_upgrade_dtest:
+ utests_long_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
- resource_class: xlarge
+ resource_class: medium
working_directory: ~/
shell: /bin/bash -eo pipefail -l
parallelism: 25
@@ -311,82 +310,133 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_UPGRADE_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_UPGRADE_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_UPGRADE_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_UPGRADE_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_LONG_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_LONG_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS_LONG} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if false; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if true; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_UPGRADE_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
- fi
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant long-testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
- store_test_results:
- path: /tmp/results
+ path: /tmp/results/repeated_utests/output
- store_artifacts:
- path: /tmp/dtest
- destination: dtest
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
- store_artifacts:
- path: ~/cassandra-dtest/logs
- destination: dtest_logs
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -402,28 +452,29 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_repeated_dtest:
+ j8_unit_tests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
- resource_class: large
+ resource_class: medium
working_directory: ~/
shell: /bin/bash -eo pipefail -l
parallelism: 25
@@ -431,73 +482,247 @@ jobs:
- attach_workspace:
at: /home/cassandra
- run:
- name: Clone Cassandra dtest Repository (via git)
- command: |
- git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
- - run:
- name: Configure virtualenv and python Dependencies
+ name: Log Environment Information
command: |
- # note, this should be super quick as all dependencies should be pre-installed in the docker image
- # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
- # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
- # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
- pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
- pip3 freeze
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
- run:
- name: Run repeated Python dtest
+ name: Repeatedly run new or modifed JUnit tests
no_output_timeout: 15m
command: |
- if [ "${REPEATED_DTEST_NAME}" == "<nil>" ]; then
- echo "Repeated dtest name hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" == "<nil>" ]; then
- echo "Repeated dtest count hasn't been defined, exiting without running any test"
- elif [ "${REPEATED_DTEST_COUNT}" -le 0 ]; then
- echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
- else
-
- # Calculate the number of test iterations to be run by the current parallel runner.
- # Since we are running the same test multiple times there is no need to use `circleci tests split`.
- count=$((${REPEATED_DTEST_COUNT} / CIRCLE_NODE_TOTAL))
- if (($CIRCLE_NODE_INDEX < (${REPEATED_DTEST_COUNT} % CIRCLE_NODE_TOTAL))); then
- count=$((count+1))
- fi
-
- if (($count <= 0)); then
- echo "No tests to run in this runner"
- else
- echo "Running ${REPEATED_DTEST_NAME} $count times"
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
- source ~/env3.6/bin/activate
- export PATH=$JAVA_HOME/bin:$PATH
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_UTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
- java -version
- cd ~/cassandra-dtest
- mkdir -p /tmp/dtest
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_UTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
- echo "env: $(env)"
- echo "** done env"
- mkdir -p /tmp/results/dtests
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
- stop_on_failure_arg=""
- if ${REPEATED_DTEST_STOP_ON_FAILURE}; then
- stop_on_failure_arg="-x"
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
fi
- vnodes_args=""
- if ${REPEATED_DTEST_VNODES}; then
- vnodes_args="--use-vnodes --num-tokens=16"
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
fi
- upgrade_arg=""
- if false; then
- upgrade_arg="--execute-upgrade-tests"
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
fi
- # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
- set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir ${REPEATED_DTEST_NAME} | tee /tmp/dtest/stdout.txt
- fi
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant testsome $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_upgrade_dtests_repeat:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: xlarge
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 25
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Clone Cassandra dtest Repository (via git)
+ command: |
+ git clone --single-branch --branch $DTEST_BRANCH --depth 1 $DTEST_REPO ~/cassandra-dtest
+ - run:
+ name: Configure virtualenv and python Dependencies
+ command: |
+ # note, this should be super quick as all dependencies should be pre-installed in the docker image
+ # if additional dependencies were added to requirmeents.txt and the docker image hasn't been updated
+ # we'd have to install it here at runtime -- which will make things slow, so do yourself a favor and
+ # rebuild the docker image! (it automatically pulls the latest requirements.txt on build)
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+ pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
+ pip3 freeze
+ - run:
+ name: Run repeated Python dtest
+ no_output_timeout: 15m
+ command: |
+ if [ "${REPEATED_UPGRADE_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_UPGRADE_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_UPGRADE_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_UPGRADE_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if false; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if true; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
fi
- store_test_results:
path: /tmp/results
@@ -522,22 +747,23 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
build:
@@ -619,31 +845,32 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_dtests-no-vnodes:
+ j8_dtests_vnode_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: large
working_directory: ~/
shell: /bin/bash -eo pipefail -l
- parallelism: 50
+ parallelism: 25
steps:
- attach_workspace:
at: /home/cassandra
@@ -663,21 +890,69 @@ jobs:
pip3 install --upgrade -r ~/cassandra-dtest/requirements.txt
pip3 freeze
- run:
- name: Determine Tests to Run (j8_without_vnodes)
- no_output_timeout: 5m
- command: "# reminder: this code (along with all the steps) is independently executed on every circle container\n# so the goal here is to get the circleci script to return the tests *this* container will run\n# which we do via the `circleci` cli tool.\n\ncd cassandra-dtest\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\n\nif [ -n '' ]; then\n export \nfi\n\necho \"***Collected DTests (j8_without_vnodes)***\"\nset -eo pipefail && ./run_dtests.py --skip-resource-in [...]
- - run:
- name: Run dtests (j8_without_vnodes)
+ name: Run repeated Python dtest
no_output_timeout: 15m
- command: "echo \"cat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\"\ncat /tmp/split_dtest_tests_j8_without_vnodes_final.txt\n\nsource ~/env3.6/bin/activate\nexport PATH=$JAVA_HOME/bin:$PATH\nif [ -n '' ]; then\n export \nfi\n\njava -version\ncd ~/cassandra-dtest\nmkdir -p /tmp/dtest\n\necho \"env: $(env)\"\necho \"** done env\"\nmkdir -p /tmp/results/dtests\n# we need the \"set -o pipefail\" here so that the exit code that circleci will actually use is from pytest and not [...]
+ command: |
+ if [ "${REPEATED_DTESTS}" == "<nil>" ]; then
+ echo "Repeated dtest name hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" == "<nil>" ]; then
+ echo "Repeated dtest count hasn't been defined, exiting without running any test"
+ elif [ "${REPEATED_DTESTS_COUNT}" -le 0 ]; then
+ echo "Repeated dtest count is lesser or equals than zero, exiting without running any test"
+ else
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ # Since we are running the same test multiple times there is no need to use `circleci tests split`.
+ count=$((${REPEATED_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ if (($count <= 0)); then
+ echo "No tests to run in this runner"
+ else
+ echo "Running ${REPEATED_DTESTS} $count times"
+
+ source ~/env3.6/bin/activate
+ export PATH=$JAVA_HOME/bin:$PATH
+
+ java -version
+ cd ~/cassandra-dtest
+ mkdir -p /tmp/dtest
+
+ echo "env: $(env)"
+ echo "** done env"
+ mkdir -p /tmp/results/dtests
+
+ tests_arg=$(echo ${REPEATED_DTESTS} | sed -e "s/,/ /g")
+
+ stop_on_failure_arg=""
+ if ${REPEATED_TESTS_STOP_ON_FAILURE}; then
+ stop_on_failure_arg="-x"
+ fi
+
+ vnodes_args=""
+ if true; then
+ vnodes_args="--use-vnodes --num-tokens=16"
+ fi
+
+ upgrade_arg=""
+ if false; then
+ upgrade_arg="--execute-upgrade-tests --upgrade-target-version-only --upgrade-version-selection all"
+ fi
+
+ # we need the "set -o pipefail" here so that the exit code that circleci will actually use is from pytest and not the exit code from tee
+ set -o pipefail && cd ~/cassandra-dtest && pytest $vnodes_args --count=$count $stop_on_failure_arg $upgrade_arg --log-cli-level=DEBUG --junit-xml=/tmp/results/dtests/pytest_result.xml -s --cassandra-dir=/home/cassandra/cassandra --keep-test-dir $tests_arg | tee /tmp/dtest/stdout.txt
+ fi
+ fi
- store_test_results:
path: /tmp/results
- store_artifacts:
path: /tmp/dtest
- destination: dtest_j8_without_vnodes
+ destination: dtest
- store_artifacts:
path: ~/cassandra-dtest/logs
- destination: dtest_j8_without_vnodes_logs
+ destination: dtest_logs
environment:
- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- ANT_HOME: /usr/share/ant
@@ -693,25 +968,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_upgradetests-no-vnodes:
+ j8_upgrade_dtests:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: xlarge
@@ -808,25 +1084,26 @@ jobs:
- DTEST_BRANCH: trunk
- CCM_MAX_HEAP_SIZE: 1024M
- CCM_HEAP_NEWSIZE: 256M
- - REPEATED_UTEST_TARGET: testsome
- - REPEATED_UTEST_CLASS: null
- - REPEATED_UTEST_METHODS: null
- - REPEATED_UTEST_COUNT: 100
- - REPEATED_UTEST_STOP_ON_FAILURE: false
- - REPEATED_DTEST_NAME: null
- - REPEATED_DTEST_VNODES: false
- - REPEATED_DTEST_COUNT: 100
- - REPEATED_DTEST_STOP_ON_FAILURE: false
- - REPEATED_UPGRADE_DTEST_NAME: null
- - REPEATED_UPGRADE_DTEST_COUNT: 100
- - REPEATED_UPGRADE_DTEST_STOP_ON_FAILURE: false
- - REPEATED_JVM_UPGRADE_DTEST_CLASS: null
- - REPEATED_JVM_UPGRADE_DTEST_METHODS: null
- - REPEATED_JVM_UPGRADE_DTEST_COUNT: 100
- - REPEATED_JVM_UPGRADE_DTEST_STOP_ON_FAILURE: false
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
- j8_unit_tests:
+ j8_jvm_upgrade_dtests_repeat:
docker:
- image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
resource_class: medium
@@ -836,25 +1113,6 @@ jobs:
steps:
- attach_workspace:
at: /home/cassandra
- - run:
- name: Determine unit Tests to Run
- command: |
- # reminder: this code (along with all the steps) is independently executed on every circle container
- # so the goal here is to get the circleci script to return the tests *this* container will run
- # which we do via the `circleci` cli tool.
-
- rm -fr ~/cassandra-dtest/upgrade_tests
- echo "***java tests***"
-
- # get all of our unit test filenames
- set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
-
- # split up the unit tests into groups based on the number of containers we have
- set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
- set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
- cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
- no_output_timeout: 15m
- run:
name: Log Environment Information
command: |
@@ -878,7 +1136,8 @@ jobs:
which java
java -version
- run:
- name: Run Unit Tests (testclasslist)
+ name: Repeatedly run new or modifed JUnit tests
+ no_output_timeout: 15m
command: |
set -x
export PATH=$JAVA_HOME/bin:$PATH
@@ -887,19 +1146,707 @@ jobs:
if [ -d ~/dtest_jars ]; then
cp ~/dtest_jars/dtest* /tmp/cassandra/build/
fi
- test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
- if [ -z "$test_timeout" ]; then
- test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+
+ # Calculate the number of test iterations to be run by the current parallel runner.
+ count=$((${REPEATED_JVM_UPGRADE_DTESTS_COUNT} / CIRCLE_NODE_TOTAL))
+ if (($CIRCLE_NODE_INDEX < (${REPEATED_JVM_UPGRADE_DTESTS_COUNT} % CIRCLE_NODE_TOTAL))); then
+ count=$((count+1))
+ fi
+
+ # Put manually specified tests and automatically detected tests together, removing duplicates
+ tests=$(echo ${REPEATED_JVM_UPGRADE_DTESTS} | sed -e "s/<nil>//" | sed -e "s/ //" | tr "," "\n" | tr " " "\n" | sort -n | uniq -u)
+ echo "Tests to be repeated: ${tests}"
+
+ # Run each test class as many times as requested.
+ exit_code="$?"
+ for test in $tests; do
+
+ # Split class and method names from the test name
+ if [[ $test =~ "#" ]]; then
+ class=${test%"#"*}
+ method=${test#*"#"}
+ else
+ class=$test
+ method=""
+ fi
+
+ # Prepare the -Dtest.name argument.
+ # It can be the fully qualified class name or the short class name, depending on the target.
+ if [[ $target == "test" || \
+ $target == "test-cdc" || \
+ $target == "test-compression" || \
+ $target == "test-system-keyspace-directory" || \
+ $target == "long-test" ]]; then
+ name_arg="-Dtest.name=${class##*.}"
+ else
+ name_arg="-Dtest.name=$class"
+ fi
+
+ # Prepare the -Dtest.methods argument, which is optional
+ if [ $method == "" ]; then
+ methods_arg=""
+ else
+ methods_arg="-Dtest.methods=$method"
+ fi
+
+ for i in $(seq -w 1 $count); do
+ echo "Running test $test, iteration $i of $count"
+
+ # run the test
+ status="passes"
+ if !( set -o pipefail && \
+ ant test-jvm-dtest-some $name_arg $methods_arg -Dno-build-test=true | \
+ tee stdout.txt \
+ ); then
+ status="fails"
+ exit_code=1
+ fi
+
+ # move the stdout output file
+ dest=/tmp/results/repeated_utests/stdout/${status}/${i}
+ mkdir -p $dest
+ mv stdout.txt $dest/${test}.txt
+
+ # move the XML output files
+ source=build/test/output
+ dest=/tmp/results/repeated_utests/output/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # move the log files
+ source=build/test/logs
+ dest=/tmp/results/repeated_utests/logs/${status}/${i}
+ mkdir -p $dest
+ if [[ -d $source && -n "$(ls $source)" ]]; then
+ mv $source/* $dest/
+ fi
+
+ # maybe stop iterations on test failure
+ if [[ ${REPEATED_TESTS_STOP_ON_FAILURE} = true ]] && (( $exit_code > 0 )); then
+ break
+ fi
+ done
+ done
+ (exit ${exit_code})
+ - store_test_results:
+ path: /tmp/results/repeated_utests/output
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/stdout
+ destination: stdout
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/output
+ destination: junitxml
+ - store_artifacts:
+ path: /tmp/results/repeated_utests/logs
+ destination: logs
+ environment:
+ - JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - ANT_HOME: /usr/share/ant
+ - LANG: en_US.UTF-8
+ - KEEP_TEST_DIR: true
+ - DEFAULT_DIR: /home/cassandra/cassandra-dtest
+ - PYTHONIOENCODING: utf-8
+ - PYTHONUNBUFFERED: true
+ - CASS_DRIVER_NO_EXTENSIONS: true
+ - CASS_DRIVER_NO_CYTHON: true
+ - CASSANDRA_SKIP_SYNC: true
+ - DTEST_REPO: https://github.com/apache/cassandra-dtest.git
+ - DTEST_BRANCH: trunk
+ - CCM_MAX_HEAP_SIZE: 1024M
+ - CCM_HEAP_NEWSIZE: 256M
+ - REPEATED_TESTS_STOP_ON_FAILURE: false
+ - REPEATED_UTESTS: null
+ - REPEATED_UTESTS_COUNT: 500
+ - REPEATED_UTESTS_LONG: null
+ - REPEATED_UTESTS_LONG_COUNT: 100
+ - REPEATED_JVM_DTESTS: null
+ - REPEATED_JVM_DTESTS_COUNT: 500
+ - REPEATED_JVM_UPGRADE_DTESTS: null
+ - REPEATED_JVM_UPGRADE_DTESTS_COUNT: 500
+ - REPEATED_DTESTS: null
+ - REPEATED_DTESTS_COUNT: 500
+ - REPEATED_UPGRADE_DTESTS: null
+ - REPEATED_UPGRADE_DTESTS_COUNT: 25
+ - REPEATED_ANT_TEST_TARGET: testsome
+ - REPEATED_ANT_TEST_CLASS: null
+ - REPEATED_ANT_TEST_METHODS: null
+ - REPEATED_ANT_TEST_COUNT: 500
+ - JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ - JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+ j8_unit_tests:
+ docker:
+ - image: apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
+ resource_class: medium
+ working_directory: ~/
+ shell: /bin/bash -eo pipefail -l
+ parallelism: 25
+ steps:
+ - attach_workspace:
+ at: /home/cassandra
+ - run:
+ name: Determine unit Tests to Run
+ command: |
+ # reminder: this code (along with all the steps) is independently executed on every circle container
+ # so the goal here is to get the circleci script to return the tests *this* container will run
+ # which we do via the `circleci` cli tool.
+
+ rm -fr ~/cassandra-dtest/upgrade_tests
+ echo "***java tests***"
+
+ # get all of our unit test filenames
+ set -eo pipefail && circleci tests glob "$HOME/cassandra/test/unit/**/*.java" > /tmp/all_java_unit_tests.txt
+
+ # split up the unit tests into groups based on the number of containers we have
+ set -eo pipefail && circleci tests split --split-by=timings --timings-type=filename --index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
+ set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | sed "s;^/home/cassandra/cassandra/test/unit/;;g" | grep "Test\.java$" > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
+ cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
+ no_output_timeout: 15m
+ - run:
+ name: Log Environment Information
+ command: |
+ echo '*** id ***'
+ id
+ echo '*** cat /proc/cpuinfo ***'
+ cat /proc/cpuinfo
+ echo '*** free -m ***'
+ free -m
+ echo '*** df -m ***'
+ df -m
+ echo '*** ifconfig -a ***'
+ ifconfig -a
+ echo '*** uname -a ***'
+ uname -a
+ echo '*** mount ***'
+ mount
+ echo '*** env ***'
+ env
+ echo '*** java ***'
+ which java
+ java -version
+ - run:
+ name: Run Unit Tests (testclasslist)
+ command: |
+ set -x
+ export PATH=$JAVA_HOME/bin:$PATH
+ time mv ~/cassandra /tmp
+ cd /tmp/cassandra
+ if [ -d ~/dtest_jars ]; then
+ cp ~/dtest_jars/dtest* /tmp/cassandra/build/
+ fi
+ test_timeout=$(grep 'name="test.unit.timeout"' build.xml | awk -F'"' '{print $4}' || true)
+ if [ -z "$test_timeout" ]; then
+ test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' '{print $4}')
+ fi
+ ant testclasslist -Dtest.timeout="$test_timeout" -Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt -Dtest.classlistprefix=unit
+ no_output_timeout: 15m
+ - store_test_results:
... 1615 lines suppressed ...
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org