You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by mc...@apache.org on 2022/05/03 19:36:03 UTC
[cassandra-builds] branch trunk updated (b830c21 -> d2d15ce)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
from b830c21 Revert "ninja-fix – disable arm64 builds until INFRA-22930 is resolved properly"
new 3d31501 Use the git clone reference feature in jenkins, pointing each split's git clone to the first split
new d2d15ce Update trunk to version 4.2
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build-scripts/cassandra-test.sh | 2 +-
docker/almalinux-image.docker | 1 +
docker/buster-image.docker | 1 +
docker/centos8-image.docker | 1 +
docker/jessie-image.docker | 1 +
docker/testing/ubuntu1910_j11.docker | 133 ---------------------
.../testing/ubuntu1910_j11_w_dependencies.docker | 39 ------
.../testing/ubuntu2004_j11_w_dependencies.docker | 3 +
jenkins-dsl/cassandra_job_dsl_seed.groovy | 13 +-
9 files changed, 16 insertions(+), 178 deletions(-)
delete mode 100644 docker/testing/ubuntu1910_j11.docker
delete mode 100644 docker/testing/ubuntu1910_j11_w_dependencies.docker
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
[cassandra-builds] 01/02: Use the git clone reference feature in jenkins, pointing each split's git clone to the first split
Posted by mc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
commit 3d3150124c3d6aef94f580a0f97531aa9c427f65
Author: Mick Semb Wever <mc...@apache.org>
AuthorDate: Sun Apr 10 20:33:58 2022 +0200
Use the git clone reference feature in jenkins, pointing each split's git clone to the first split
Also clean the workspace when jobs are done.
patch by Mick Semb Wever; reviewed by Ekaterina Dimitrova for CASSANDRA-17589
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index c2e27a7..43b6493 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -272,7 +272,7 @@ matrixJob('Cassandra-template-dtest-matrix') {
cleanAfterCheckout()
cloneOption {
shallow(false)
- reference('.')
+ reference('../1')
honorRefspec(true)
noTags(true)
timeout(maxJobHours * 60)
@@ -541,6 +541,7 @@ cassandraBranches.each {
}
}
}
+ cleanWs()
}
}
}
@@ -643,6 +644,7 @@ cassandraBranches.each {
}
}
}
+ cleanWs()
}
}
}
@@ -716,6 +718,7 @@ cassandraBranches.each {
}
}
}
+ cleanWs()
}
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
[cassandra-builds] 02/02: Update trunk to version 4.2
Posted by mc...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
commit d2d15cee9c417e502e29a1a92ded6efa31b92707
Author: Mick Semb Wever <mc...@apache.org>
AuthorDate: Mon May 2 15:59:41 2022 +0200
Update trunk to version 4.2
Ddocker testing images to include cassandra-4.2 (delete unused ubuntu1910).
Build dtest jar for 4.2 too.
Add 4.1 ci-cassandra.a.o jobs.
Increase git's http.postBuffer (common CI problem).
patch by Mick Semb Wever; reviewed by Ekaterina Dimitrova for CASSANDRA-17589
---
build-scripts/cassandra-test.sh | 2 +-
docker/almalinux-image.docker | 1 +
docker/buster-image.docker | 1 +
docker/centos8-image.docker | 1 +
docker/jessie-image.docker | 1 +
docker/testing/ubuntu1910_j11.docker | 133 ---------------------
.../testing/ubuntu1910_j11_w_dependencies.docker | 39 ------
.../testing/ubuntu2004_j11_w_dependencies.docker | 3 +
jenkins-dsl/cassandra_job_dsl_seed.groovy | 8 +-
9 files changed, 12 insertions(+), 177 deletions(-)
diff --git a/build-scripts/cassandra-test.sh b/build-scripts/cassandra-test.sh
index e83778c..bb9b9d3 100755
--- a/build-scripts/cassandra-test.sh
+++ b/build-scripts/cassandra-test.sh
@@ -38,7 +38,7 @@ _build_all_dtest_jars() {
cd $TMP_DIR
until git clone --quiet --depth 1 --no-single-branch https://github.com/apache/cassandra.git cassandra-dtest-jars ; do echo "git clone failed… trying again… " ; done
cd cassandra-dtest-jars
- for branch in cassandra-2.2 cassandra-3.0 cassandra-3.11 cassandra-4.0 trunk; do
+ for branch in cassandra-2.2 cassandra-3.0 cassandra-3.11 cassandra-4.0 cassandra-4.1 trunk; do
git checkout $branch
ant realclean
ant jar dtest-jar
diff --git a/docker/almalinux-image.docker b/docker/almalinux-image.docker
index 01c266a..3b7b17a 100644
--- a/docker/almalinux-image.docker
+++ b/docker/almalinux-image.docker
@@ -47,6 +47,7 @@ RUN mkdir -p $RPM_BUILD_DIR/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
# Clone Cassandra and cache maven artifacts
ARG CASSANDRA_GIT_URL=https://github.com/apache/cassandra.git
+RUN git config --global http.postBuffer 524288000
RUN git clone ${CASSANDRA_GIT_URL} ${CASSANDRA_DIR}
WORKDIR $CASSANDRA_DIR
diff --git a/docker/buster-image.docker b/docker/buster-image.docker
index 04a3e87..c24167c 100644
--- a/docker/buster-image.docker
+++ b/docker/buster-image.docker
@@ -48,6 +48,7 @@ USER build
# clone Cassandra and cache maven artifacts
ARG CASSANDRA_GIT_URL=https://github.com/apache/cassandra.git
+RUN git config --global http.postBuffer 524288000
RUN git clone ${CASSANDRA_GIT_URL} ${CASSANDRA_DIR}
WORKDIR ${CASSANDRA_DIR}
diff --git a/docker/centos8-image.docker b/docker/centos8-image.docker
index 01ddd0b..226dcec 100644
--- a/docker/centos8-image.docker
+++ b/docker/centos8-image.docker
@@ -47,6 +47,7 @@ RUN mkdir -p $RPM_BUILD_DIR/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
# Clone Cassandra and cache maven artifacts
ARG CASSANDRA_GIT_URL=https://github.com/apache/cassandra.git
+RUN git config --global http.postBuffer 524288000
RUN git clone ${CASSANDRA_GIT_URL} ${CASSANDRA_DIR}
WORKDIR $CASSANDRA_DIR
diff --git a/docker/jessie-image.docker b/docker/jessie-image.docker
index 1ae10cc..b564a15 100644
--- a/docker/jessie-image.docker
+++ b/docker/jessie-image.docker
@@ -42,6 +42,7 @@ USER build
# clone Cassandra and cache maven artifacts
ARG CASSANDRA_GIT_URL=https://github.com/apache/cassandra.git
+RUN git config --global http.postBuffer 524288000
RUN git clone ${CASSANDRA_GIT_URL} ${CASSANDRA_DIR}
WORKDIR ${CASSANDRA_DIR}
diff --git a/docker/testing/ubuntu1910_j11.docker b/docker/testing/ubuntu1910_j11.docker
deleted file mode 100644
index 0449852..0000000
--- a/docker/testing/ubuntu1910_j11.docker
+++ /dev/null
@@ -1,133 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-FROM ubuntu:19.10
-MAINTAINER Apache Cassandra <de...@cassandra.apache.org>
-
-# configure apt to retry downloads
-RUN echo 'APT::Acquire::Retries "9";' > /etc/apt/apt.conf.d/80-retries
-
-# install our python dependencies and some other stuff we need
-# libev4 libev-dev are for the python driver / libssl-dev is for python3.6
-
-RUN export DEBIAN_FRONTEND=noninteractive && \
- sed -i -re 's/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list && \
- apt-get update && \
- apt-get install -y --no-install-recommends software-properties-common apt-utils vim
-
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y git-core python2.7 python3-pip python3.8 python3.8-venv python3.8-dev net-tools libev4 libev-dev wget gcc libssl-dev
-
-# need to install Python 3.6 as well
-RUN cd /opt && wget https://www.python.org/ftp/python/3.6.10/Python-3.6.10.tgz && \
- tar xzf Python-3.6.10.tgz && cd Python-3.6.10 && \
- ./configure --enable-optimizations && \
- make altinstall && \
- cp /opt/Python-3.6.10/python /usr/bin/python3.6
-
-RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
-RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.8 3
-RUN python3.6 -m pip install --upgrade pip
-RUN python3.7 -m pip install --upgrade pip
-RUN python3.8 -m pip install --upgrade pip
-
-# solves warning: "jemalloc shared library could not be preloaded to speed up memory allocations"
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y --no-install-recommends libjemalloc2
-
-# install dumb-init as minimal init system
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y dumb-init
-
-# generate locales for the standard en_US.UTF8 value we use for testing
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y locales && \
- locale-gen en_US.UTF-8
-
-# as we only need the requirements.txt file from the dtest repo, let's just get it from GitHub as a raw asset
-# so we can avoid needing to clone the entire repo just to get this file
-ADD https://raw.githubusercontent.com/apache/cassandra-dtest/trunk/requirements.txt /opt
-RUN chmod 0644 /opt/requirements.txt
-
-# now setup python via virtualenv with all of the python dependencies we need according to requirements.txt
-RUN pip3 install virtualenv
-RUN pip3 install --upgrade wheel
-
-# openjdk + ant
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y --no-install-recommends openjdk-8-jdk openjdk-11-jdk ant ant-optional
-
-# make Java 8 the default executable (we use to run all tests against Java 8)
-RUN update-java-alternatives --set java-1.8.0-openjdk-$(dpkg --print-architecture)
-
-# setup our user -- if we don't do this docker will default to root and Cassandra will fail to start
-# as we appear to have a check that the user isn't starting Cassandra as root
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get install sudo && \
- adduser --disabled-password --gecos "" cassandra && \
- echo "cassandra ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/cassandra && \
- chmod 0440 /etc/sudoers.d/cassandra
-
-# fix up permissions on the cassandra home dir
-RUN chown -R cassandra:cassandra /home/cassandra
-
-# switch to the cassandra user... we are all done running things as root
-USER cassandra
-ENV HOME /home/cassandra
-WORKDIR /home/cassandra
-
-# Add environment variables for Ant and Java and add them to the PATH
-RUN echo 'export ANT_HOME=/usr/share/ant' >> /home/cassandra/.bashrc && \
- echo 'export JAVA8_HOME=/usr/lib/jvm/java-8-openjdk-$(dpkg --print-architecture)' >> /home/cassandra/.bashrc && \
- echo 'export JAVA11_HOME=/usr/lib/jvm/java-11-openjdk-$(dpkg --print-architecture)' >> /home/cassandra/.bashrc && \
- echo 'export JAVA_HOME=$JAVA8_HOME' >> /home/cassandra/.bashrc
-
-ENV ANT_HOME=/usr/share/ant
-
-# run pip commands and setup virtualenv (note we do this after we switch to cassandra user so we
-# setup the virtualenv for the cassandra user and not the root user by accident) for Python 3.6/3.7/3.8
-RUN virtualenv --python=python3.6 env3.6
-RUN chmod +x env3.6/bin/activate
-RUN /bin/bash -c "source ~/env3.6/bin/activate && pip3 install Cython && pip3 install -r /opt/requirements.txt && pip3 freeze --user"
-
-RUN virtualenv --python=python3.7 env3.7
-RUN chmod +x env3.7/bin/activate
-RUN /bin/bash -c "source ~/env3.7/bin/activate && pip3 install Cython && pip3 install -r /opt/requirements.txt && pip3 freeze --user"
-
-RUN virtualenv --python=python3.8 env3.8
-RUN chmod +x env3.8/bin/activate
-RUN /bin/bash -c "source ~/env3.8/bin/activate && pip3 install Cython && pip3 install -r /opt/requirements.txt && pip3 freeze --user"
-
-# we need to make SSH less strict to prevent various dtests from failing when they attempt to
-# git clone a given commit/tag/etc
-# upgrading node1 to github:apache/18cdd391ec27d16daf775f928902f5a421c415e3
-# git@github.com:apache/cassandra.git github:apache/18cdd391ec27d16daf775f928902f5a421c415e3
-# 23:47:08,993 ccm INFO Cloning Cassandra...
-# The authenticity of host 'github.com (192.30.253.112)' can't be established.
-# RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
-# Are you sure you want to continue connecting (yes/no)?
-RUN mkdir ~/.ssh
-RUN echo 'Host *\n UserKnownHostsFile /dev/null\n StrictHostKeyChecking no' > ~/.ssh/config
-RUN chown cassandra:cassandra ~/.ssh
-RUN chown cassandra:cassandra ~/.ssh/config
-RUN chmod 600 ~/.ssh/config
-
-# mark "/tmp" as a volume so it will get mounted as an ext4 mount and not
-# the stupid aufs/CoW stuff that the actual docker container mounts will have.
-# we've been seeing 3+ minute hangs when calling sync on an aufs backed mount
-# so it greatly makes tests flaky as things can hang basically anywhere
-VOLUME ["/tmp"]
diff --git a/docker/testing/ubuntu1910_j11_w_dependencies.docker b/docker/testing/ubuntu1910_j11_w_dependencies.docker
deleted file mode 100644
index fa4fb55..0000000
--- a/docker/testing/ubuntu1910_j11_w_dependencies.docker
+++ /dev/null
@@ -1,39 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# base things off the testing image without dependencies warmed up
-FROM apache/cassandra-testing-ubuntu1910-java11
-MAINTAINER Apache Cassandra <de...@cassandra.apache.org>
-
-USER cassandra
-ENV HOME /home/cassandra
-WORKDIR /home/cassandra
-
-# Fetch the maven dependencies in advance since this tends to fail at runtime
-ARG CASSANDRA_GIT_URL=https://gitbox.apache.org/repos/asf/cassandra.git
-RUN /bin/bash -c "git clone ${CASSANDRA_GIT_URL} ~/cassandra && \
- cd ~/cassandra && ant resolver-dist-lib && \
- git checkout origin/cassandra-4.0 && ant resolver-dist-lib && \
- git checkout origin/cassandra-3.11 && ant resolver-dist-lib && \
- git checkout origin/cassandra-3.0 && ant resolver-dist-lib && \
- git checkout origin/cassandra-2.2 && ant maven-ant-tasks-retrieve-build && \
- git checkout origin/cassandra-2.1 && ant maven-ant-tasks-retrieve-build && \
- rm -fr ~/cassandra"
-
-# Initialize the CCM git repo as well as this also can fail to clone
-RUN /bin/bash -c "source ~/env3.6/bin/activate && \
- ccm create -n 1 -v git:trunk test && ccm remove test && \
- ccm create -n 1 -v git:cassandra-4.0 test && ccm remove test && \
- ccm create -n 1 -v git:cassandra-3.11 test && ccm remove test && \
- ccm create -n 1 -v git:cassandra-3.0 test && ccm remove test && \
- ccm create -n 1 -v git:cassandra-2.2 test && ccm remove test && \
- ccm create -n 1 -v git:cassandra-2.1 test && ccm remove test"
diff --git a/docker/testing/ubuntu2004_j11_w_dependencies.docker b/docker/testing/ubuntu2004_j11_w_dependencies.docker
index d55fca9..f79ffec 100644
--- a/docker/testing/ubuntu2004_j11_w_dependencies.docker
+++ b/docker/testing/ubuntu2004_j11_w_dependencies.docker
@@ -20,8 +20,10 @@ WORKDIR /home/cassandra
# Fetch the maven dependencies in advance since this tends to fail at runtime
ARG CASSANDRA_GIT_URL=https://gitbox.apache.org/repos/asf/cassandra.git
+RUN git config --global http.postBuffer 524288000
RUN /bin/bash -c "git clone ${CASSANDRA_GIT_URL} ~/cassandra && \
cd ~/cassandra && ant resolver-dist-lib && \
+ git checkout origin/cassandra-4.1 && ant resolver-dist-lib && \
git checkout origin/cassandra-4.0 && ant resolver-dist-lib && \
git checkout origin/cassandra-3.11 && ant resolver-dist-lib && \
git checkout origin/cassandra-3.0 && ant resolver-dist-lib && \
@@ -32,6 +34,7 @@ RUN /bin/bash -c "git clone ${CASSANDRA_GIT_URL} ~/cassandra && \
# Initialize the CCM git repo as well as this also can fail to clone
RUN /bin/bash -c "source ~/env3.6/bin/activate && \
ccm create -n 1 -v git:trunk test && ccm remove test && \
+ ccm create -n 1 -v git:cassandra-4.1 test && ccm remove test && \
ccm create -n 1 -v git:cassandra-4.0 test && ccm remove test && \
ccm create -n 1 -v git:cassandra-3.11 test && ccm remove test && \
ccm create -n 1 -v git:cassandra-3.0 test && ccm remove test && \
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 43b6493..ff0ce71 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -55,7 +55,7 @@ if(binding.hasVariable("CASSANDRA_DTEST_GIT_URL")) {
}
def buildDescStr = 'REF = ${GIT_BRANCH} <br /> COMMIT = ${GIT_COMMIT}'
// Cassandra active branches
-def cassandraBranches = ['cassandra-2.2', 'cassandra-3.0', 'cassandra-3.11', 'cassandra-4.0', 'trunk']
+def cassandraBranches = ['cassandra-2.2', 'cassandra-3.0', 'cassandra-3.11', 'cassandra-4.0', 'cassandra-4.1', 'trunk']
if(binding.hasVariable("CASSANDRA_BRANCHES")) {
cassandraBranches = "${CASSANDRA_BRANCHES}".split(",")
}
@@ -377,7 +377,7 @@ cassandraBranches.each {
disabled(false)
using('Cassandra-template-artifacts')
axes {
- if (branchName == 'trunk' || branchName == 'cassandra-4.0') {
+ if (!(branchName ==~ /cassandra-[2-3].\d+/)) {
jdk('jdk_1.8_latest','jdk_11_latest')
} else {
jdk('jdk_1.8_latest')
@@ -450,7 +450,7 @@ cassandraBranches.each {
println("Skipping ${targetName} on branch ${branchName}")
// Skip tests that don't exist before cassandra-4.0
- } else if ((targetName == 'fqltool-test') && ((branchName == 'cassandra-2.2') || (branchName == 'cassandra-3.0') || (branchName == 'cassandra-3.11'))) {
+ } else if (targetName == 'fqltool-test' && branchName ==~ /cassandra-[2-3].\d+/) {
println("Skipping ${targetName} on branch ${branchName}")
} else {
@@ -466,7 +466,7 @@ cassandraBranches.each {
_testSplits = "/${testSplits}"
}
// jvm-dtest-upgrade would require mixed JDK compilations to support JDK11+
- if ((branchName == 'trunk' || branchName == 'cassandra-4.0') && targetName != 'jvm-dtest-upgrade') {
+ if ((!(branchName ==~ /cassandra-[2-3].\d+/)) && targetName != 'jvm-dtest-upgrade') {
jdk(jdkLabel,'jdk_11_latest')
} else {
jdk(jdkLabel)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org