You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2022/02/08 03:55:07 UTC
[GitHub] [couchdb-ci] kocolosk opened a new pull request #33: Update process for multi-arch images, add builder Action
kocolosk opened a new pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33
This PR documents the process to build multi-arch container images with the Docker `buildx` plugin. It also creates a GitHub Action that allows building these images directly from the GitHub UI instead of messing with the `build.sh` script.
Currently the images get posted to the GitHub Container Registry instead of Docker Hub. I did this intentionally as it seems worthwhile to evaluate using GHCR for this "internal" CouchDB use case instead of Docker Hub. But it really is just an evaluation. Happy to entertain feedback.
I am interested in the idea of setting up some event triggering so that e.g. new images get published when a new Erlang version is released and then get picked up automatically by Jenkins. For now though the Action is just a manually invoked one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] kocolosk commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r815318707
##########
File path: README.md
##########
@@ -59,31 +65,38 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building a cross-architecture Docker image
-This only works from an `x86_64` build host.
-
-First, configure your machine with the correct dependencies to build multi-arch binaries:
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
+multi-architecture container images with a single command invocation. Docker
+Desktop ships with buildx support, but you'll need to create a new builder to
+use it:
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+docker buildx create --use
Review comment:
Fixed in 3b8001b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] kocolosk commented on pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
kocolosk commented on pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#issuecomment-1052168905
@nickva I changed the strategy here to add new `buildx` targets and preserve the support for the original approach of generating architecture-specific images.
The end effect is two different sets of images. One could do the work to generate a multi-architecture manifest from the single architecture images manually, following the pattern from the [couchdb-docker build.sh script](https://github.com/apache/couchdb-docker/blob/efa6e93389c0d1b7980894f9059fb3a5775864b1/build.sh#L147-L164), but I didn't include that here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] kocolosk commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r801828355
##########
File path: README.md
##########
@@ -59,31 +65,38 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building a cross-architecture Docker image
-This only works from an `x86_64` build host.
-
-First, configure your machine with the correct dependencies to build multi-arch binaries:
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
+multi-architecture container images with a single command invocation. Docker
+Desktop ships with buildx support, but you'll need to create a new builder to
+use it:
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+docker buildx create --use
Review comment:
Hm, OK. Fair request.
One issue is that the previous process only produced arch-specific images and didn't do the work to produce a single multi-architecture manifest. That's fixable.
I'll also have to see about reworking the dockerfiles to allow for the old $CONTAINERARCH approach. Let me see if there's a simple way to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] kocolosk commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r815318707
##########
File path: README.md
##########
@@ -59,31 +65,38 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building a cross-architecture Docker image
-This only works from an `x86_64` build host.
-
-First, configure your machine with the correct dependencies to build multi-arch binaries:
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
+multi-architecture container images with a single command invocation. Docker
+Desktop ships with buildx support, but you'll need to create a new builder to
+use it:
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+docker buildx create --use
Review comment:
Fixed in ed7c197
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] nickva commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r801802170
##########
File path: README.md
##########
@@ -59,31 +65,38 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building a cross-architecture Docker image
-This only works from an `x86_64` build host.
-
-First, configure your machine with the correct dependencies to build multi-arch binaries:
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
+multi-architecture container images with a single command invocation. Docker
+Desktop ships with buildx support, but you'll need to create a new builder to
+use it:
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+docker buildx create --use
Review comment:
`buildx` doesn't seem to be available in the open source packages (Ubuntu LTS 20.04 for instance) and only if installed from Docker Desktop, or from the deb package installed from docker.io directly. Let's leave instructions how to do it without docker.io closed source bits around as well. For example, when generating the latest package I didn't use buildx and used the instructions which are being deleted in the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] nickva commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r801802170
##########
File path: README.md
##########
@@ -59,31 +65,38 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building a cross-architecture Docker image
-This only works from an `x86_64` build host.
-
-First, configure your machine with the correct dependencies to build multi-arch binaries:
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
+multi-architecture container images with a single command invocation. Docker
+Desktop ships with buildx support, but you'll need to create a new builder to
+use it:
```
-docker run --privileged --rm tonistiigi/binfmt --install all
+docker buildx create --use
Review comment:
`buildx` doesn't seem to be available in the open source packages (Ubuntu LTS 20.04 for instance) and only from if installed from Docker Desktop, or from the deb package installed from docker.io directly. Let's leave instruction how to do it without docker.io closed source bits around as well. For example, when generating the latest package I didn't use buildx and used the instructions which are being deleted in the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-ci] nickva commented on a change in pull request #33: Update process for multi-arch images, add builder Action
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #33:
URL: https://github.com/apache/couchdb-ci/pull/33#discussion_r815480435
##########
File path: README.md
##########
@@ -59,9 +58,34 @@ the [kerl](https://github.com/kerl/kerl) build system, and installs them to
`/usr/local/kerl` for activation before builds. This version is intended for use
in standard CI runs, such as for pull requests.
-# Building a cross-architecture Docker image
+## Building images for other architectures
+
+### Multi-arch images with Docker Buildx
+
+We can use Docker's
+[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate
Review comment:
A tiny nit, but since it tripped me up, it might be worth mentioning that the open source docker.io package from Ubuntu default repositories do not support buildx (at least in LTS 20.04). So to use it, it is necessary to directly install the package from docker.io. Overall I am not particularly keen on standardizing on proprietary bits from Docker.
reference: https://docs.docker.com/buildx/working-with-buildx/#linux-packages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org