You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2022/12/06 13:07:20 UTC
[GitHub] [couchdb] konrad-ohms commented on issue #3517: Performance regression on CouchDB v3 while using custom reduce function
konrad-ohms commented on issue #3517:
URL: https://github.com/apache/couchdb/issues/3517#issuecomment-1339319235
Hey @janl, thanks for picking the issue up again.
I tried to rebuild but hit a dependency problem in the docker build, I guess I might have missed overwriting certain dependency versions somewhere...
By any chance, did your CI system put out a docker image I could just use for running the validation?
This is what I tried:
```bash
$ git clone https://github.com/apache/couchdb-ci.git
$ cd couchdb-ci/
$ ./build.sh platform debian-buster
...
==> Downloading Elixir from https://repo.hex.pm/builds/elixir/v1.13.4-otp-23.zip
--> dbab54520ba
STEP 12/14: RUN echo "jenkins ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/jenkins
--> c59f6b6aca0
STEP 13/14: USER jenkins
--> 39e19399d38
STEP 14/14: CMD ["/bin/bash"]
COMMIT apache/couchdbci-debian:buster-erlang-23.3.4.15
--> 617bc6f2f04
Successfully tagged localhost/apache/couchdbci-debian:buster-erlang-23.3.4.15
617bc6f2f04e049eb6ab418828de3239b06ce1a74b756a031dd5f33ea342c05b
$ podman run --name couchdb-fix-3517 -it localhost/apache/couchdbci-debian:buster-erlang-23.3.4.15
jenkins@b730dc2b6711:/$ cd ~
jenkins@b730dc2b6711:~$ whoami
jenkins
jenkins@b730dc2b6711:~$ pwd
/home/jenkins
jenkins@b730dc2b6711:~$ git clone --depth=1 -b fix/3517-part7 https://github.com/apache/couchdb.git
Cloning into 'couchdb'...
remote: Enumerating objects: 1508, done.
remote: Counting objects: 100% (1508/1508), done.
remote: Compressing objects: 100% (1309/1309), done.
remote: Total 1508 (delta 290), reused 625 (delta 138), pack-reused 0
Receiving objects: 100% (1508/1508), 4.95 MiB | 1.64 MiB/s, done.
Resolving deltas: 100% (290/290), done.
jenkins@b730dc2b6711:~$ cd couchdb/
jenkins@b730dc2b6711:~/couchdb$ git log
commit 05628e891b5145c09e429eb79a21631cddf63c43 (grafted
, HEAD -> fix/3517-part7, origin/fix/3517-part7
)
Author: Jan Lehnardt <ja...@apache.org>
Date: Tue Dec 6 12:50:15 2022 +0100
fix(3517): super-simplistic fix to avoid costly AST transforms when they are
not needed
jenkins@b730dc2b6711:~/couchdb$ ./configure
...
jenkins@b730dc2b6711:~/couchdb$ make release
...
WARN: Missing plugins: [covertool]
WARN: Missing plugins: [pc]
==> rel (generate)
WARN: 'generate' command does not apply to directory /home/jenkins/couchdb
... done
You can now copy the rel/couchdb directory anywhere on your system.
Start CouchDB with ./bin/couchdb from within that directory.
jenkins@b730dc2b6711:~/couchdb$ echo $?
0
jenkins@b730dc2b6711:~/couchdb$ make dist
...
Done: apache-couchdb-3.2.2-05628e8.tar.gz
```
I copied back the build results and packaged them up:
```bash
$ podman cp couchdb-fix-3517:/home/jenkins/couchdb/apache-couchdb-3.2.2-05628e8.tar.gz .
$ git clone https://github.com/apache/couchdb-pkg.git
Cloning into 'couchdb-pkg'...
remote: Enumerating objects: 860, done.
remote: Counting objects: 100% (234/234), done.
remote: Compressing objects: 100% (143/143), done.
remote: Total 860 (delta 103), reused 173 (delta 75), pack-reused 626
Receiving objects: 100% (860/860), 25.34 MiB | 10.53 MiB/s, done.
Resolving deltas: 100% (397/397), done.
$ cd couchdb-pkg/
$ podman tag apache/couchdbci-debian:buster-erlang-23.3.4.15 docker.io/couchdbdev/debian-buster-erlang-23.3.4.15
$ ERLANGVERSION=23.3.4.15 ./build.sh couch debian-buster /home/ohmsk/Downloads/couchdb-test/apache-couchdb-3.2.2-05628e8.tar.gz
...
$ ls -lah pkgs/couch/debian-buster/
total 35M
drwxrwxrwx. 2 ohmsk ohmsk 207 Dec 6 13:48 .
drwxrwxrwx. 3 ohmsk ohmsk 45 Dec 6 13:48 ..
-rwxr-xr-x. 1 ohmsk ohmsk 7.6K Dec 6 13:48 couchdb_3.2.2-05628e8~buster_amd64.buildinfo
-rwxr-xr-x. 1 ohmsk ohmsk 1.5K Dec 6 13:48 couchdb_3.2.2-05628e8~buster_amd64.changes
-rwxr-xr-x. 1 ohmsk ohmsk 27M Dec 6 13:48 couchdb_3.2.2-05628e8~buster_amd64.deb
-rwxr-xr-x. 1 ohmsk ohmsk 8.6M Dec 6 13:48 couchdb-dbgsym_3.2.2-05628e8~buster_amd64.deb
```
And include the binaries in the image:
```bash
$ git clone https://github.com/apache/couchdb-docker.git
Cloning into 'couchdb-docker'...
remote: Enumerating objects: 1093, done.
remote: Counting objects: 100% (201/201), done.
remote: Compressing objects: 100% (97/97), done.
remote: Total 1093 (delta 124), reused 164 (delta 100), pack-reused 892
Receiving objects: 100% (1093/1093), 265.20 KiB | 498.00 KiB/s, done.
Resolving deltas: 100% (550/550), done.
$ cd couchdb-docker/
$ ls
2.3.1 3.1.2-ubi 3.2.0-ubi-clouseau build.sh LICENSE
2.3.1-ubi 3.1.2-ubi-clouseau 3.2.1 dev README.md
3.1.2 3.2.0 3.2.2 dev-cluster
$ cd 3.2.2/
$ cp -r ~/Downloads/couchdb-test/couchdb-pkg/pkgs/couch/debian-buster/ .
$ ls debian-buster/
couchdb_3.2.2-05628e8~buster_amd64.buildinfo
couchdb_3.2.2-05628e8~buster_amd64.changes
couchdb_3.2.2-05628e8~buster_amd64.deb
couchdb-dbgsym_3.2.2-05628e8~buster_amd64.deb
```
I adjusted the Dockerfile again to just copy deb packages into the image to validate the patch:
```bash
$ git diff Dockerfile
diff --git a/3.2.2/Dockerfile b/3.2.2/Dockerfile
index 62177b5..ec8e679 100644
--- a/3.2.2/Dockerfile
+++ b/3.2.2/Dockerfile
@@ -60,6 +60,9 @@ RUN . /etc/os-release; \
echo "deb [signed-by=/usr/share/keyrings/couchdb-archive-keyring.gpg] https://apache.jfrog.io/artifactory/couchdb-deb/ ${VERSION_CODENAME} main" | \
tee /etc/apt/sources.list.d/couchdb.list >/dev/null
+COPY debian-buster/couchdb_3.2.2-05628e8~buster_amd64.deb /tmp/packages/
+COPY debian-buster/couchdb-dbgsym_3.2.2-05628e8~buster_amd64.deb /tmp/packages/
+
# https://github.com/apache/couchdb-pkg/blob/master/debian/README.Debian
RUN set -eux; \
apt-get update; \
@@ -67,7 +70,7 @@ RUN set -eux; \
echo "couchdb couchdb/mode select none" | debconf-set-selections; \
# we DO want recommends this time
DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-downgrades --allow-remove-essential --allow-change-held-packages \
- couchdb="$COUCHDB_VERSION"~bullseye \
+ /tmp/packages/* \
; \
# Undo symlinks to /var/log and /var/lib
rmdir /var/lib/couchdb /var/log/couchdb; \
$ podman build -t "couchdb:3.2.2-patch-jan-fix-3517" .
STEP 1/20: FROM debian:bullseye-slim
STEP 2/20: LABEL maintainer="CouchDB Developers dev@couchdb.apache.org"
--> Using cache 09f4784df121998589bdcab66be2643748d0e2db2f49a87c19b52c0a771837f3
--> 09f4784df12
STEP 3/20: RUN groupadd -g 5984 -r couchdb && useradd -u 5984 -d /opt/couchdb -g couchdb couchdb
--> Using cache 081f8b18791f9a3459ad1f40641f7eb4590a3da68e42a46a636d61f58aff0e4d
--> 081f8b18791
STEP 4/20: RUN set -ex; apt-get update; apt-get install -y --no-install-recommends apt-transport-https ca-certificates dirmngr gnupg ; rm -rf /var/lib/apt/lists/*
--> Using cache e683e09e78629b6bff7b85d7f8fd2835c840a299ecbf16d848ca71b69e628edd
--> e683e09e786
STEP 5/20: RUN set -eux; apt-get update; apt-get install -y --no-install-recommends gosu tini; rm -rf /var/lib/apt/lists/*; gosu nobody true; tini --version
--> Using cache b0c10606eae929b003afec60212eef33809a96f5acd81bb037f304c1dc0bd7b7
--> b0c10606eae
STEP 6/20: ENV GPG_COUCH_KEY 390EF70BB1EA12B2773962950EE62FB37A00258D
--> Using cache ad3a2134d39a73a793dea21fcd2d178cc4a9fc80d35e7fc952871db417c91c52
--> ad3a2134d39
STEP 7/20: RUN set -eux; apt-get update; apt-get install -y curl; export GNUPGHOME="$(mktemp -d)"; curl -fL -o keys.asc https://couchdb.apache.org/repo/keys.asc; gpg --batch --import keys.asc; gpg --batch --export "${GPG_COUCH_KEY}" > /usr/share/keyrings/couchdb-archive-keyring.gpg; command -v gpgconf && gpgconf --kill all || :; rm -rf "$GNUPGHOME"; apt-key list; apt purge -y --autoremove curl; rm -rf /var/lib/apt/lists/*
--> Using cache b6575178b4ca6bb472eab38e3b517993a5010c8fb367c1ac5cb9884ae4828412
--> b6575178b4c
STEP 8/20: ENV COUCHDB_VERSION 3.2.2-1
--> Using cache 7177e466f1a1164eaf7f381a033978e5ec1fdb8ca3da819c30a2af691d35305c
--> 7177e466f1a
STEP 9/20: RUN . /etc/os-release; echo "deb [signed-by=/usr/share/keyrings/couchdb-archive-keyring.gpg] https://apache.jfrog.io/artifactory/couchdb-deb/ ${VERSION_CODENAME} main" | tee /etc/apt/sources.list.d/couchdb.list >/dev/null
--> Using cache 3f504a04eb8d62bd2b1e81dec865b5ccd22cec5f2f2f8793f21a468fa18ee0a9
--> 3f504a04eb8
STEP 10/20: COPY debian-buster/couchdb_3.2.2-05628e8~buster_amd64.deb /tmp/packages/
--> Using cache 58f61fe809b5da2a974b71d729203dc269713c9297e01ab2f70a354bb946ad06
--> 58f61fe809b
STEP 11/20: COPY debian-buster/couchdb-dbgsym_3.2.2-05628e8~buster_amd64.deb /tmp/packages/
--> Using cache 756d9ecfccba323e98d6781562881f596af0590801896f59deb2b3f4d9f3790e
--> 756d9ecfccb
STEP 12/20: RUN set -eux; apt-get update; echo "couchdb couchdb/mode select none" | debconf-set-selections; DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-downgrades --allow-remove-essential --allow-change-held-packages /tmp/packages/* ; rmdir /var/lib/couchdb /var/log/couchdb; rm /opt/couchdb/data /opt/couchdb/var/log; mkdir -p /opt/couchdb/data /opt/couchdb/var/log; chown couchdb:couchdb /opt/couchdb/data /opt/couchdb/var/log; chmod 777 /opt/couchdb/data /opt/couchdb/var/log; rm /opt/couchdb/etc/default.d/10-filelog.ini; find /opt/couchdb \! \( -user couchdb -group couchdb \) -exec chown -f couchdb:couchdb '{}' +; find /opt/couchdb/etc -type d ! -perm 0755 -exec chmod -f 0755 '{}' +; find /opt/couchdb/etc -type f ! -perm 0644 -exec chmod -f 0644 '{}' +; chmod -f 0777 /opt/couchdb/etc/local.d; rm -rf /var/lib/apt/lists/*;
+ apt-get update
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8184 kB]
Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [208 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [14.6 kB]
Get:7 https://apache.jfrog.io/artifactory/couchdb-deb bullseye InRelease [5155 B]
Get:8 https://apache.jfrog.io/artifactory/couchdb-deb bullseye/main amd64 Packages [5242 B]
Fetched 8625 kB in 1s (6869 kB/s)
Reading package lists...
+ echo couchdb couchdb/mode select none
+ debconf-set-selections
+ DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-downgrades --allow-remove-essential --allow-change-held-packages /tmp/packages/couchdb-dbgsym_3.2.2-05628e8~buster_amd64.deb /tmp/packages/couchdb_3.2.2-05628e8~buster_amd64.deb
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
couchdb : Depends: libmozjs-60-0 but it is not installable
Depends: libicu63 (>= 63.1-1~) but it is not installable
E: Unable to correct problems, you have held broken packages.
Error: error building at STEP "RUN set -eux; apt-get update; echo "couchdb couchdb/mode select none" | debconf-set-selections; DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-downgrades --allow-remove-essential --allow-change-held-packages /tmp/packages/* ; rmdir /var/lib/couchdb /var/log/couchdb; rm /opt/couchdb/data /opt/couchdb/var/log; mkdir -p /opt/couchdb/data /opt/couchdb/var/log; chown couchdb:couchdb /opt/couchdb/data /opt/couchdb/var/log; chmod 777 /opt/couchdb/data /opt/couchdb/var/log; rm /opt/couchdb/etc/default.d/10-filelog.ini; find /opt/couchdb \! \( -user couchdb -group couchdb \) -exec chown -f couchdb:couchdb '{}' +; find /opt/couchdb/etc -type d ! -perm 0755 -exec chmod -f 0755 '{}' +; find /opt/couchdb/etc -type f ! -perm 0644 -exec chmod -f 0644 '{}' +; chmod -f 0777 /opt/couchdb/etc/local.d; rm -rf /var/lib/apt/lists/*;": error while running runtime: exit status 100
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org