You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by mc...@apache.org on 2021/08/09 20:15:47 UTC
[cassandra-builds] branch trunk updated: Deploy multi-architecture
(amd+arm) docker testing images
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new cb77538 Deploy multi-architecture (amd+arm) docker testing images
cb77538 is described below
commit cb77538bc28c546ccac1078ffb658d313142e43e
Author: Mick Semb Wever <mc...@apache.org>
AuthorDate: Mon Aug 9 14:03:13 2021 +0200
Deploy multi-architecture (amd+arm) docker testing images
patch by Mick Semb Wever; reviewed by Brandon Williams for CASSANDRA-16833
---
docker/testing/README.md | 28 +++++++++++++---------------
docker/testing/ubuntu2004_j11.docker | 4 ++--
jenkins-dsl/cassandra_job_dsl_seed.groovy | 14 --------------
3 files changed, 15 insertions(+), 31 deletions(-)
diff --git a/docker/testing/README.md b/docker/testing/README.md
index 195f95e..96ce247 100644
--- a/docker/testing/README.md
+++ b/docker/testing/README.md
@@ -5,33 +5,31 @@ Docker files in this directory are used to build images used by ci-cassandra.apa
* Base image for Linux distribution to use for testing (e.g. `ubuntu1910_j11.docker`)
* Caching image that contains git sources, maven and ccm dependencies
-## Building Images
+## Preparing Multi-Architecture Building
+
+ docker buildx create --name mybuilder
+ docker buildx use mybuilder
+ docker buildx inspect
+
+
+This is based on the documentation found [here](https://www.docker.com/blog/multi-arch-images/).
+
+## Building and Publishing Images
Build images from the parent directory using the following commands. Change tag (`-t`) as needed (prefix and current date):
Base image:
-`docker build -t apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d") -t apache/cassandra-testing-ubuntu2004-java11:latest -f ubuntu2004_j11.docker .`
+ docker buildx build --platform linux/amd64,linux/arm64 -t apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d") -t apache/cassandra-testing-ubuntu2004-java11:latest -f ubuntu2004_j11.docker --push .
Caching image:
-`docker build --no-cache -t apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date +"%Y%m%d") -t apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest -f ubuntu2004_j11_w_dependencies.docker .`
+ docker buildx build --platform linux/amd64,linux/arm64 --no-cache -t apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date +"%Y%m%d") -t apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest -f ubuntu2004_j11_w_dependencies.docker --push .
Please make sure to always tag also by date, so we can go back to that version in case anything breaks after the next update!
-## Publishing Images
-
We are using Docker Hub for storing published images. See [Quick Start Guide](https://docs.docker.com/docker-hub/) on how to setup docker.
-Push both image references:
-
-```
-docker push apache/cassandra-testing-ubuntu2004-java11:$(date +"%Y%m%d")
-docker push apache/cassandra-testing-ubuntu2004-java11:latest
-docker push apache/cassandra-testing-ubuntu2004-java11-w-dependencies:$(date +"%Y%m%d")
-docker push apache/cassandra-testing-ubuntu2004-java11-w-dependencies:latest
-```
-
## Updating circleci.yml
-Check if the correct image is used in `cassandra_job_dsl_seed.groovy` and `.circleci/config*.yml`. It should either be set to the date dervived tag created above, or `:latest`.
+Check if the correct image is used in `cassandra_job_dsl_seed.groovy` and `.circleci/config*.yml`. It should either be set to the date derived tag created above, or `:latest`.
diff --git a/docker/testing/ubuntu2004_j11.docker b/docker/testing/ubuntu2004_j11.docker
index 48bc812..5540715 100644
--- a/docker/testing/ubuntu2004_j11.docker
+++ b/docker/testing/ubuntu2004_j11.docker
@@ -26,11 +26,11 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
RUN export DEBIAN_FRONTEND=noninteractive && \
add-apt-repository -y ppa:deadsnakes/ppa && \
apt-get update && \
- apt-get install -y git-core python2.7 python3-pip \
+ apt-get install -y git-core python2.7 python-dev python3-pip \
python3.6 python3.6-venv python3.6-dev \
python3.7 python3.7-venv python3.7-dev \
python3.8 python3.8-venv python3.8-dev \
- net-tools libev4 libev-dev wget gcc libssl-dev
+ net-tools libev4 libev-dev wget gcc libssl-dev libxml2 libxslt1-dev
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.7 3
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index 8c69f96..36809f3 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -564,13 +564,6 @@ cassandraBranches.each {
node / scm / branches / 'hudson.plugins.git.BranchSpec' / name(branchName)
}
steps {
- if (arch == "-arm64") {
- shell("""
- # docker image has to be built on arm64 (they are not currently published to dockerhub)
- cd cassandra-builds/docker/testing ;
- docker build -t ${dtestDockerImage}:latest -f ubuntu2004_j11.docker .
- """)
- }
shell("""
./cassandra-builds/build-scripts/cassandra-dtest-pytest-docker.sh apache ${branchName} https://github.com/apache/cassandra-dtest.git trunk ${buildsRepo} ${buildsBranch} ${dtestDockerImage} ${targetName} \${split}/${splits} ;
wget --retry-connrefused --waitretry=1 "\${BUILD_URL}/timestamps/?time=HH:mm:ss&timeZone=UTC&appendLog" -qO - > console.log.xz || echo wget failed
@@ -1020,13 +1013,6 @@ archs.each {
echo "cassandra-builds at: `git -C cassandra-builds log -1 --pretty=format:'%h %an %ad %s'`" ;
echo "Cassandra-devbranch-${targetArchName}) cassandra: `git log -1 --pretty=format:'%h %an %ad %s'`" > Cassandra-devbranch-${targetArchName}.head ;
""")
- if (arch == "-arm64") {
- shell("""
- # docker image has to be built on arm64 (they are not currently published to dockerhub)
- cd cassandra-builds/docker/testing ;
- docker build -t \$DOCKER_IMAGE:latest -f ubuntu2004_j11.docker .
- """)
- }
shell("""
./cassandra-builds/build-scripts/cassandra-dtest-pytest-docker.sh \$REPO \$BRANCH \$DTEST_REPO \$DTEST_BRANCH ${buildsRepo} ${buildsBranch} \$DOCKER_IMAGE ${targetName} \${split}/${splits} ;
wget --retry-connrefused --waitretry=1 "\${BUILD_URL}/timestamps/?time=HH:mm:ss&timeZone=UTC&appendLog" -qO - > console.log.xz || echo wget failed
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org