You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2022/02/07 14:50:18 UTC
[GitHub] [couchdb-pkg] kocolosk opened a new pull request #90: Update Erlang version + some other modernizations
kocolosk opened a new pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90
## Overview
This PR uses the new multi-architecture container images that we have in CI, relying on the `--platform` flag to select the appropriate architecture at runtime. It also upgrades to Debian 11 for the base OS that we use for multi-platform packaging by default (and the one we use for Docker images), and bumps the underlying Erlang version to Erlang 23.
Updating to Erlang 23 should fix #86 .
I'm planning to upload new packages that incorporate these changes and bump the package version number / release number appropriately. We don't currently have packaging version numbers parameterized in the code so I figured I'd just create a "3.2.1" branch and a `3.2.1-1` tag with the package numbers bumped off the side of the main branch for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] nickva commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
nickva commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1031940929
> And by the way I was referring to the couchdb-pkg repo as the location for the 3.2.1 branch and 3.2.1-1 tag, not the couchdb repo itself, in case that wasn't clear.
You're right, I did get confused there. Thanks for clarifying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on a change in pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#discussion_r800837706
##########
File path: build.sh
##########
@@ -31,10 +31,10 @@ SCRIPTPATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
DEBIANS="debian-stretch debian-buster debian-bullseye"
UBUNTUS="ubuntu-bionic ubuntu-focal"
CENTOSES="centos-7 centos-8"
-XPLAT_BASE="debian-buster"
-XPLAT_ARCHES="arm64v8 ppc64le"
+XPLAT_BASE="debian-bullseye"
+XPLAT_ARCHES="arm64 ppc64le"
Review comment:
This StackOverflow answer provides some extra detail:
https://stackoverflow.com/a/70889505/4797770
It looks like `containerd` will happily normalize `arm64/v8` as `arm64`, but `arm64v8` is treated as a separate architecture entirely.
I also found that while we would use `arm64v8` in the name of the single-arch images that we had been publishing, the architecture was actually labeled as `arm64`.
Bottom line: because we are identifying the image to use via the `--platform` flag instead of the prefix in the single-arch image name, we need to use `arm64` or `arm64/v8` and not `arm64v8` here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on a change in pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#discussion_r800738780
##########
File path: build.sh
##########
@@ -31,10 +31,10 @@ SCRIPTPATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
DEBIANS="debian-stretch debian-buster debian-bullseye"
UBUNTUS="ubuntu-bionic ubuntu-focal"
CENTOSES="centos-7 centos-8"
-XPLAT_BASE="debian-buster"
-XPLAT_ARCHES="arm64v8 ppc64le"
+XPLAT_BASE="debian-bullseye"
+XPLAT_ARCHES="arm64 ppc64le"
Review comment:
I might be able to revert this and change the architecture name in the underlying image, e.g. https://hub.docker.com/layers/apache/couchdbci-debian/bullseye-erlang-23/images/sha256-d72059db579e0497a44076c22ef3c32755ad49f0c2766c78f7550fff36e9fb4f?context=explore. But it's also not clear to me what the "right" answer is, e.g. I see lots of "arm64/v8" in Docker Hub. Regardless, it seems to be recognized as an ARM image just ine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk merged pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk merged pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] nickva commented on a change in pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
nickva commented on a change in pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#discussion_r800906791
##########
File path: build.sh
##########
@@ -31,10 +31,10 @@ SCRIPTPATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
DEBIANS="debian-stretch debian-buster debian-bullseye"
UBUNTUS="ubuntu-bionic ubuntu-focal"
CENTOSES="centos-7 centos-8"
-XPLAT_BASE="debian-buster"
-XPLAT_ARCHES="arm64v8 ppc64le"
+XPLAT_BASE="debian-bullseye"
+XPLAT_ARCHES="arm64 ppc64le"
Review comment:
That makes sense, thanks for digging in and and figuring it out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on a change in pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on a change in pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#discussion_r800733127
##########
File path: build.sh
##########
@@ -88,13 +88,15 @@ build-couch() {
docker run \
--mount type=bind,src=${SCRIPTPATH},dst=/home/jenkins/couchdb-pkg \
-u 0 -w /home/jenkins/couchdb-pkg \
+ --platform linux/amd64 \
Review comment:
Omitting this seems to cause Docker to pick up whatever image it might have in the cache and run it, even if it's not the native architecture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1031890385
Hi @nickva, yes, I hacked in metadata updates at d59bd6cca5dcddb2959b9539a9ae2af4fde3aba5, I only meant that we don't allow to drive the package versioning via environment variables currently. Convention seems to be that Debian packages can have an optional version 0, while RPMs start at release 1, so the packages I have ready to publish are e.g.
> couchdb_3.2.1-1~bullseye_amd64.deb
and
> couchdb-3.2.1-2.el7.x86_64.rpm
and
> couchdb_3.2.1-1~bionic_amd64.deb
That last one doesn't seem to quite follow the Ubuntu scheme exactly but maybe that's OK.
Yes, I agree with the notion of being able to publish multiple package versions for a given CouchDB release, e.g. with updated Erlang builds. We haven't relied on this much to date.
I'm not as familiar with specific guidance on the Docker side. It's complicated by the fact that a CI pipeline rebuilding images on a regular basis e.g. to pick up security updates in the underlying OS could have significantly more frequent updates. A 1:1 mapping between the container image tag and the CouchDB package could make sense, although you'd run into a little bit of friction because I'd expect you also want to tag an updated 3.2.1-1 package with the 3.2.1 tag, since unlike apt and yum Docker won't automatically pull the 3.2.1-1 image for you when you ask for 3.2.1.
> Just to clarify, that would be 3.2.1 branch off of the 3.2.1 tag not off of the current 3.x? Something like:
Yes, exactly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1032114663
https://stackoverflow.com/questions/65071806/how-do-i-run-a-cluster-of-nodes-with-both-otp-23-1-and-r15b03-1 goes into a bit of detail on the distribution protocol compatibility and doesn't indicate any issues for a mixed 20/23 cluster, although admittedly they just checked connectivity and not much in the way of richer RPCs.
I'm not sure what to do about the performance QA piece. We do need to upgrade; Erlang 20 hasn't seen a patch release in nearly two years and has known bugs that are preventing replication with one of the more popular CouchDB-compatible endpoints out there. I suspect our user community would be better off running newer releases instead of waiting for Cloudant to successfully upgrade its estate, but demonstrating that with any level of rigor is not a small project.
If you're saying, "pin each CouchDB version to a single Erlang major version for convenience packages", I do think that makes some sense. Even if we did adopt that policy (which I floated on dev@), one counter I have in this case is that CouchDB 3.2.1 is kinda/sorta not fully released yet from a packaging perspective, e.g. we never published any blog post on it or updated the homepage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] nickva commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
nickva commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1033168514
> If you're saying, "pin each CouchDB version to a single Erlang major version for convenience packages", I do think that makes some sense. Even if we did adopt that policy (which I floated on dev@), one counter I have in this case is that CouchDB 3.2.1 is kinda/sorta not fully released yet from a packaging perspective, e.g. we never published any blog post on it or updated the homepage.
I guess, or we'd decide release by release based on befits vs risk. I was thinking from the point of view of a user who updated to 3.2.1-0 and setup their repo to auto-upgrade then, they do to 3.2.1-1 but now their they might get slightly different performance characteristics or even breakage. However, in this case, like we saw there are already serious breakages so I after thinking more on it, we're probably fine updating.
> I suspect our user community would be better off running newer releases instead of waiting for Cloudant to successfully upgrade its estate, but demonstrating that with any level of rigor is not a small project.
I guess going by the point that we haven't actually officially announced 3.2.1 and only published the binary package (my fault for jumping the gun there), I see just going with 23 and updating. It strengthens the case for that at least.
It looks like Cloudant is in the testing process and running benchmarks as we speak, we can always return back and make another package and pick another version or update some default vm.args flags or something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] nickva commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
nickva commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1031806728
> We don't currently have packaging version numbers parameterized in the code
Good point that we don't have package / docker specific versioning. Something like `3.2.1-1` might work. Technically, the Docker image might have it's own `-1`, if we updated any docker specific things (like here https://github.com/apache/couchdb-docker/pull/215#issuecomment-1029998115), and the binary packages might get their own if, say, we bumped Erlang versions, or updated pre/post install scripts. The RPMs could have a `-1` but DEBs might not.
Noticed Ubuntu have a similar issue for their package versioning, and they have a scheme like: `A.B.C-XubuntuY` with `A.B.C` is the base version of the package, `X` is the Debian version and `Y` the Ubuntu specific one.
> ... create a "3.2.1" branch and a 3.2.1-1 tag with the package numbers bumped off the side of the main branch for now.
Just to clarify, that would be 3.2.1 branch off of the 3.2.1 tag not off of the current 3.x? Something like:
```
3.x
...
3.2.1 [branch][tag]
...
3.2.1-1 [tag]
```
I hope at least after https://github.com/apache/couchdb/pull/3889 is reviewed and merges, and with all improvement like the sharded index server, replicator target re-creation handling, and change feed rewind fixes, we could probably have a good reason to roll out a 3.3 as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [couchdb-pkg] kocolosk commented on pull request #90: Update Erlang version + some other modernizations
Posted by GitBox <gi...@apache.org>.
kocolosk commented on pull request #90:
URL: https://github.com/apache/couchdb-pkg/pull/90#issuecomment-1031904184
And by the way I was referring to the `couchdb-pkg` repo as the location for the `3.2.1` branch and `3.2.1-1` tag, not the `couchdb` repo itself, in case that wasn't clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org