You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by wo...@apache.org on 2017/07/19 21:46:25 UTC
[couchdb] branch jenkins-packages-2 updated (9a6f615 -> 2636a15)
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a change to branch jenkins-packages-2
in repository https://gitbox.apache.org/repos/asf/couchdb.git.
discard 9a6f615 Build repos and push to couchdb-vm2
discard c40f66d0 Fix _local_docs end-point
add 860f23c Fix _local_docs end-point
add 99c195b fix double export of wait_updated/3
add 0bea3a4 Fix test teardown
add 3523bab Rename unused variables
add 3e14510 Use hashed password when we create admin in test
add 6a559cf Use random name for _users db in test
add bd0ee45 Fix trailing whitespace issues
add 483f59e Merge pull request #691 from cloudant/3367-fix-test-case
add 37d7fad Use test_util:stop_config in mem3_util_test
add b0a2fe8 Merge pull request #693 from cloudant/use-stop_sync-in-mem3-test
add c8bf463 Remove get_details replicator job gen_server call
add 0f46cde Simplify regression test for COUCHDB-1283
add 27147cd Fix cancellation race in replication.js tests
add dfa14c0 TMP: Add debug logging for failed assertion
new 825d0ad Fix _local_docs end-point
new 2636a15 Build repos and push to couchdb-vm2
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9a6f615)
\
N -- N -- N refs/heads/jenkins-packages-2 (2636a15)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
Jenkinsfile | 182 +++++++++------------
src/chttpd/test/chttpd_security_tests.erl | 34 ++--
src/couch/src/couch_changes.erl | 3 +-
src/couch/test/couchdb_compaction_daemon_tests.erl | 7 +
src/couch/test/couchdb_views_tests.erl | 170 +++----------------
.../src/couch_replicator_scheduler_job.erl | 3 -
.../test/couch_replicator_compact_tests.erl | 10 --
src/mango/src/mango_idx_text.erl | 56 ++++---
src/mem3/test/mem3_util_test.erl | 4 +-
test/javascript/tests/replication.js | 2 +-
10 files changed, 161 insertions(+), 310 deletions(-)
--
To stop receiving notification emails like this one, please contact
['"commits@couchdb.apache.org" <co...@couchdb.apache.org>'].
[couchdb] 02/02: Build repos and push to couchdb-vm2
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch jenkins-packages-2
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 2636a1582a4b1d20be477951532d7518303749f6
Author: Joan Touzet <jo...@atypical.net>
AuthorDate: Wed Jul 19 03:42:39 2017 -0400
Build repos and push to couchdb-vm2
---
Jenkinsfile | 210 +++++++++++++++++++++++++++++-------------------------------
1 file changed, 102 insertions(+), 108 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 88401b8..c5405aa 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -31,24 +31,29 @@ pipeline {
image 'couchdbdev/ubuntu-14.04-erlang-default'
// https://github.com/jenkins-infra/jenkins.io/blob/master/Jenkinsfile#64
// We need the jenkins user mapped inside of the image
+ // npm config cache below is required because /home/jenkins doesn't
+ // ACTUALLY exist in the image
args '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group'
}
}
steps {
timeout(time: 15, unit: "MINUTES") {
- // npm config cache below is required because /home/jenkins doesn't
- // ACTUALLY exist in the image
sh '''
set
- rm -rf apache-couchdb-*.tar.gz
- #export npm_config_cache=$(mktemp -d)
+ rm -rf apache-couchdb-*
./configure --with-curl
make dist
- chmod -R a+w *
+ #chown -R jenkins:jenkins * || true
'''
+ }
+ }
+ post {
+ success {
stash includes: 'apache-couchdb-*.tar.gz', name: 'tarball'
archiveArtifacts artifacts: 'apache-couchdb-*.tar.gz', fingerprint: true
- deleteDir()
+ }
+ always {
+ deleteDir()
}
}
}
@@ -56,9 +61,9 @@ pipeline {
// TODO rework this once JENKINS-41334 is released
// https://issues.jenkins-ci.org/browse/JENKINS-41334
- // The builddir stuff is to prevent all 10 builds from live syncing
- // their build results to each other during the build. Moving the
- // build outside of the workdir should speed up the build process, too
+ // The builddir stuff is to prevent all the builds from live syncing
+ // their build results to each other during the build, which ACTUALLY
+ // HAPPENS. Ugh.
// we run all package builds as root inside the Docker container
@@ -68,41 +73,38 @@ pipeline {
parallel(centos6erlang183: {
node(label: 'ubuntu') {
timeout(time: 60, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/centos-6-erlang-18.3'
withDockerContainer(image: 'couchdbdev/centos-6-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
umask 0
- rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
# Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
# Build CouchDB packages
cd $builddir
git clone https://github.com/apache/couchdb-pkg
mkdir couchdb
- cd couchdb
- cp $cwd/apache-couchdb-*.tar.gz .
- tar -xf apache-couchdb-*.tar.gz
- cd ../couchdb-pkg
- # centos only
+ cp $cwd/apache-couchdb-*.tar.gz couchdb
+ tar -xf apache-couchdb-*.tar.gz -C couchdb
+ cd couchdb-pkg
platform=centos6
make $platform PLATFORM=$platform
+
+ # Cleanup & save for posterity
rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
- # CentOS variant
mv ~/rpmbuild/RPMS/x86_64/*rpm $cwd/pkgs/$platform || true
-
- # Cleanup
- cd $cwd && rm -rf $builddir
- chown -R jenkins:jenkins *
+ chown -R jenkins:jenkins $cwd/*
'''
} // withDocker
} // timeout
@@ -113,19 +115,21 @@ pipeline {
centos7erlangdefault: {
node(label: 'ubuntu') {
timeout(time: 45, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/centos-7-erlang-default'
withDockerContainer(image: 'couchdbdev/centos-7-erlang-default', args: '-e LD_LIBRARY_PATH=/usr/local/bin -e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
@@ -136,41 +140,38 @@ pipeline {
centos7erlang183: {
node(label: 'ubuntu') {
timeout(time: 60, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/centos-7-erlang-18.3'
withDockerContainer(image: 'couchdbdev/centos-7-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
umask 0
- rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
# Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
# Build CouchDB packages
cd $builddir
git clone https://github.com/apache/couchdb-pkg
mkdir couchdb
- cd couchdb
- cp $cwd/apache-couchdb-*.tar.gz .
- tar -xf apache-couchdb-*.tar.gz
- cd ../couchdb-pkg
- # centos only
+ cp $cwd/apache-couchdb-*.tar.gz couchdb
+ tar -xf apache-couchdb-*.tar.gz -C couchdb
+ cd couchdb-pkg
platform=centos7
make $platform PLATFORM=$platform
+
+ # Cleanup & save for posterity
rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
- # CentOS variant
mv ~/rpmbuild/RPMS/x86_64/*rpm $cwd/pkgs/$platform || true
-
- # Cleanup
- cd $cwd && rm -rf $builddir
- chown -R jenkins:jenkins *
+ chown -R jenkins:jenkins $cwd/*
'''
} // withDocker
} // timeout
@@ -181,19 +182,19 @@ pipeline {
ubuntu1204erlang183: {
node(label: 'ubuntu') {
timeout(time: 45, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-12.04-erlang-18.3'
withDockerContainer(image: 'couchdbdev/ubuntu-12.04-erlang-18.3', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
@@ -204,19 +205,19 @@ pipeline {
ubuntu1404erlangdefault: {
node(label: 'ubuntu') {
timeout(time: 45, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-14.04-erlang-default'
withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
@@ -227,42 +228,39 @@ pipeline {
ubuntu1404erlang183: {
node(label: 'ubuntu') {
timeout(time: 60, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-14.04-erlang-18.3'
withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
umask 0
- rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
# Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
# Build CouchDB packages
cd $builddir
git clone https://github.com/apache/couchdb-pkg
mkdir couchdb
- cd couchdb
- cp $cwd/apache-couchdb-*.tar.gz .
- tar -xf apache-couchdb-*.tar.gz
- cd ../couchdb-pkg
- # debian/ubuntu only
+ cp $cwd/apache-couchdb-*.tar.gz couchdb
+ tar -xf apache-couchdb-*.tar.gz -C couchdb
+ cd couchdb-pkg
sudo apt-get install -y libmozjs185-dev
platform=$(lsb_release -cs)
make $platform PLATFORM=$platform
+
+ # Cleanup & save for posterity
rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
- # Ubuntu/Debian variant
mv ../couchdb/*deb $cwd/pkgs/$platform || true
-
- # Cleanup
- cd $cwd && rm -rf $builddir
- chown -R jenkins:jenkins *
+ chown -R jenkins:jenkins $cwd/*
'''
} // withDocker
} // timeout
@@ -273,19 +271,19 @@ pipeline {
ubuntu1604erlangdefault: {
node(label: 'ubuntu') {
timeout(time: 45, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-16.04-erlang-default'
withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
@@ -296,42 +294,39 @@ pipeline {
ubuntu1604erlang183: {
node(label: 'ubuntu') {
timeout(time: 60, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-16.04-erlang-18.3'
withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
umask 0
- rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
# Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
# Build CouchDB packages
cd $builddir
git clone https://github.com/apache/couchdb-pkg
mkdir couchdb
- cd couchdb
- cp $cwd/apache-couchdb-*.tar.gz .
- tar -xf apache-couchdb-*.tar.gz
- cd ../couchdb-pkg
- # debian/ubuntu only
+ cp $cwd/apache-couchdb-*.tar.gz couchdb
+ tar -xf apache-couchdb-*.tar.gz -C couchdb
+ cd couchdb-pkg
sudo apt-get install -y libmozjs185-dev
platform=$(lsb_release -cs)
make $platform PLATFORM=$platform
+
+ # Cleanup & save for posterity
rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
- # Ubuntu/Debian variant
mv ../couchdb/*deb $cwd/pkgs/$platform || true
-
- # Cleanup
- cd $cwd && rm -rf $builddir
- chown -R jenkins:jenkins *
+ chown -R jenkins:jenkins $cwd/*
'''
} // withDocker
} // timeout
@@ -342,19 +337,19 @@ pipeline {
debian8erlangdefault: {
node(label: 'ubuntu') {
timeout(time: 45, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/debian-8-erlang-default'
withDockerContainer(image: 'couchdbdev/debian-8-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
@@ -365,42 +360,39 @@ pipeline {
debian8erlang183: {
node(label: 'ubuntu') {
timeout(time: 60, unit: "MINUTES") {
- unstash 'tarball'
sh 'docker pull couchdbdev/debian-8-erlang-18.3'
// must run as root because of sudo step below
withDockerContainer(image: 'couchdbdev/debian-8-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
+ sh 'rm -f apache-couchdb-*.tar.gz'
+ unstash 'tarball'
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+ mkdir -p $(COUCHDB_IO_LOG_DIR)
# Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
- ./configure --with-curl
- make all
+ #./configure --with-curl
+ #make all
#make check || (build-aux/logfile-uploader.py && false)
# Build CouchDB packages
cd $builddir
git clone https://github.com/apache/couchdb-pkg
mkdir couchdb
- cd couchdb
- cp $cwd/apache-couchdb-*.tar.gz .
- tar -xf apache-couchdb-*.tar.gz
- cd ../couchdb-pkg
- # debian/ubuntu only
+ cp $cwd/apache-couchdb-*.tar.gz couchdb
+ tar -xf apache-couchdb-*.tar.gz -C couchdb
+ cd couchdb-pkg
sudo apt-get install -y libmozjs185-dev
platform=$(lsb_release -cs)
make $platform PLATFORM=$platform
+
+ # Cleanup & save for posterity
rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
- # Ubuntu/Debian variant
mv ../couchdb/*deb $cwd/pkgs/$platform || true
-
- # Cleanup
- cd $cwd && rm -rf $builddir
- chown -R jenkins:jenkins *
+ chown -R jenkins:jenkins $cwd/*
'''
} // withDocker
} // timeout
@@ -421,13 +413,14 @@ pipeline {
// This image has the deb AND rpm repo tools installed in it
image 'couchdbdev/debian-8-base'
// We need the jenkins user mapped inside of the image
+ // We must run as root because we have to create ~/.ssh for rsync
args '-v /etc/passwd:/etc/passwd -v /etc/group:/etc/group'
label 'ubuntu'
}
}
steps {
withCredentials([file(credentialsId: 'jenkins-key', variable: 'KEY')]) {
- sh 'rm -rf pkgs'
+ sh 'rm -rf pkgs *.tar.gz'
unarchive mapping: ['pkgs/' : '.']
echo 'Building Debian repo...'
sh '''
@@ -439,7 +432,7 @@ pipeline {
echo 'Building CentOS repos...'
sh '''
cd pkgs/centos6 && createrepo --database .
- cd ../centos7 && rm js* && createrepo --database .
+ cd ../centos7 && rm -f js* && createrepo --database .
'''
echo 'rsyncing repos to couchdb-vm2...'
sh '''
@@ -448,7 +441,7 @@ pipeline {
mv couchdb-pkg/repo/dists $BRANCH_NAME/debian
mv pkgs/centos6/* $BRANCH_NAME/el6
mv pkgs/centos7/* $BRANCH_NAME/el7
- rsync -avz --delete -e "ssh -i $KEY" $BRANCH_NAME username@couchdb-vm2.apache.org:/var/www/html
+ rsync -avz --delete -e "ssh -o StrictHostKeyChecking=no -i $KEY" $BRANCH_NAME jenkins@couchdb-vm2.apache.org:/var/www/html
'''
// cronjob on couchdb-vm2 cleans up old tarballs (keeps latest 10)
echo 'rsyncing source tarball to couchdb-vm2...'
@@ -457,7 +450,8 @@ pipeline {
rm -rf $BRANCH_NAME
mkdir -p $BRANCH_NAME/source
mv apache-couchdb-*.tar.gz $BRANCH_NAME/source
- rsync -avz -e "ssh -i $KEY" $BRANCH_NAME username@couchdb-vm2.apache.org:/var/www/html
+ rsync -avz -e "ssh -o StrictHostKeyChecking=no -i $KEY" $BRANCH_NAME jenkins@couchdb-vm2.apache.org:/var/www/html
+ rm -rf $BRANCH_NAME couchdb-pkg *.tar.gz
'''
deleteDir()
} // withCredentials
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.
[couchdb] 01/02: Fix _local_docs end-point
Posted by wo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wohali pushed a commit to branch jenkins-packages-2
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 825d0add611289eb624df5ef469a86c8ae3996b0
Author: Eric Avdey <ei...@eiri.ca>
AuthorDate: Mon May 1 10:27:31 2017 -0300
Fix _local_docs end-point
This is a second attempt to fix _local_docs end-point. The previous one didn't
work on big enough btree_local, because local btree doesn't have reduction fun,
so reuse of couch_db_updater:btree_by_id_reduce/2 was crashing on a bad match
as soon as btree_local was getting kp_node. Also using full fold to calculate
total_rows value turned out to be resource expensive when a database have
significant number of local documents.
This fix avoids calculating of total_rows and offset instead always setting
them to null and also setting to null update_seq when requested, since it
doesn't have meaning in context of local documents.
A fabric module fabric_view_all_docs.erl was copied and modified as
fabric_view_local_docs.erl, because re-using it for processing of both types of
the documents was getting rather convoluted.
Jira: COUCHDB-3337
---
Jenkinsfile | 313 +++++++++++++++++++++++++++++++++++++++++++++---------------
1 file changed, 234 insertions(+), 79 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 6fbb7e7..88401b8 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -26,21 +26,25 @@ pipeline {
stage('Build') {
agent {
docker {
- /* This image has the oldest Erlang we support, 16B03 */
+ label 'ubuntu'
+ // This image has the oldest Erlang we support, 16B03
image 'couchdbdev/ubuntu-14.04-erlang-default'
- /* We need the jenkins user mapped inside of the image */
- args '-v /etc/passwd:/etc/passwd -v /etc/group:/etc/group'
+ // https://github.com/jenkins-infra/jenkins.io/blob/master/Jenkinsfile#64
+ // We need the jenkins user mapped inside of the image
+ args '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group'
}
}
steps {
timeout(time: 15, unit: "MINUTES") {
- /* npm config cache below is required because /home/jenkins doesn't
- ACTUALLY exist in the image */
- /* sh 'git clone --depth 10 https://github.com/apache/couchdb .' */
+ // npm config cache below is required because /home/jenkins doesn't
+ // ACTUALLY exist in the image
sh '''
- export npm_config_cache=$(mktemp -d)
- ./configure --with-curl
- make dist
+ set
+ rm -rf apache-couchdb-*.tar.gz
+ #export npm_config_cache=$(mktemp -d)
+ ./configure --with-curl
+ make dist
+ chmod -R a+w *
'''
stash includes: 'apache-couchdb-*.tar.gz', name: 'tarball'
archiveArtifacts artifacts: 'apache-couchdb-*.tar.gz', fingerprint: true
@@ -49,44 +53,69 @@ pipeline {
}
}
- /* TODO rework this once JENKINS-41334 is released
- https://issues.jenkins-ci.org/browse/JENKINS-41334 */
- /* The builddir stuff is to prevent all 10 builds from live syncing
- their build results to each other during the build. Moving the
- build outside of the workdir should speed up the build process too,
- though it does mean we pollute /tmp whenever a build fails. */
+ // TODO rework this once JENKINS-41334 is released
+ // https://issues.jenkins-ci.org/browse/JENKINS-41334
+
+ // The builddir stuff is to prevent all 10 builds from live syncing
+ // their build results to each other during the build. Moving the
+ // build outside of the workdir should speed up the build process, too
+
+ // we run all package builds as root inside the Docker container
+
+ // Build packages on supported platforms using esl's erlang
stage('Test') {
steps {
parallel(centos6erlang183: {
node(label: 'ubuntu') {
- timeout(time: 45, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 60, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/centos-6-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/centos-6-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin --user 0:0') {
+ withDockerContainer(image: 'couchdbdev/centos-6-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ umask 0
+ rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
+
+ # Build CouchDB packages
+ cd $builddir
+ git clone https://github.com/apache/couchdb-pkg
+ mkdir couchdb
+ cd couchdb
+ cp $cwd/apache-couchdb-*.tar.gz .
+ tar -xf apache-couchdb-*.tar.gz
+ cd ../couchdb-pkg
+ # centos only
+ platform=centos6
+ make $platform PLATFORM=$platform
+ rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
+ # CentOS variant
+ mv ~/rpmbuild/RPMS/x86_64/*rpm $cwd/pkgs/$platform || true
+
+ # Cleanup
+ cd $cwd && rm -rf $builddir
+ chown -R jenkins:jenkins *
'''
} // withDocker
} // timeout
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ deleteDir()
} // node
},
centos7erlangdefault: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 45, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/centos-7-erlang-default'
- withDockerContainer(image: 'couchdbdev/centos-7-erlang-default', args: '-e LD_LIBRARY_PATH=/usr/local/bin --user 0:0') {
+ withDockerContainer(image: 'couchdbdev/centos-7-erlang-default', args: '-e LD_LIBRARY_PATH=/usr/local/bin -e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
sh '''
cwd=$(pwd)
rm -rf /tmp/couchjslogs
@@ -97,42 +126,64 @@ pipeline {
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
} // timeout
+ deleteDir()
} // node
},
centos7erlang183: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 60, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/centos-7-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/centos-7-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin --user 0:0') {
+ withDockerContainer(image: 'couchdbdev/centos-7-erlang-18.3', args: '-e LD_LIBRARY_PATH=/usr/local/bin -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ umask 0
+ rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
+
+ # Build CouchDB packages
+ cd $builddir
+ git clone https://github.com/apache/couchdb-pkg
+ mkdir couchdb
+ cd couchdb
+ cp $cwd/apache-couchdb-*.tar.gz .
+ tar -xf apache-couchdb-*.tar.gz
+ cd ../couchdb-pkg
+ # centos only
+ platform=centos7
+ make $platform PLATFORM=$platform
+ rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
+ # CentOS variant
+ mv ~/rpmbuild/RPMS/x86_64/*rpm $cwd/pkgs/$platform || true
+
+ # Cleanup
+ cd $cwd && rm -rf $builddir
+ chown -R jenkins:jenkins *
'''
} // withDocker
} // timeout
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ deleteDir()
} // node
},
ubuntu1204erlang183: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 45, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-12.04-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/ubuntu-12.04-erlang-18.3', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/ubuntu-12.04-erlang-18.3', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
sh '''
cwd=$(pwd)
rm -rf /tmp/couchjslogs
@@ -143,19 +194,19 @@ pipeline {
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
} // timeout
+ deleteDir()
} // node
},
ubuntu1404erlangdefault: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 45, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-14.04-erlang-default'
- withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-default', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
sh '''
cwd=$(pwd)
rm -rf /tmp/couchjslogs
@@ -166,42 +217,65 @@ pipeline {
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
} // timeout
+ deleteDir()
} // node
},
ubuntu1404erlang183: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 60, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-14.04-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-18.3', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/ubuntu-14.04-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ umask 0
+ rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
+
+ # Build CouchDB packages
+ cd $builddir
+ git clone https://github.com/apache/couchdb-pkg
+ mkdir couchdb
+ cd couchdb
+ cp $cwd/apache-couchdb-*.tar.gz .
+ tar -xf apache-couchdb-*.tar.gz
+ cd ../couchdb-pkg
+ # debian/ubuntu only
+ sudo apt-get install -y libmozjs185-dev
+ platform=$(lsb_release -cs)
+ make $platform PLATFORM=$platform
+ rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
+ # Ubuntu/Debian variant
+ mv ../couchdb/*deb $cwd/pkgs/$platform || true
+
+ # Cleanup
+ cd $cwd && rm -rf $builddir
+ chown -R jenkins:jenkins *
'''
} // withDocker
} // timeout
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ deleteDir()
} // node
},
ubuntu1604erlangdefault: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 45, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-16.04-erlang-default'
- withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-default', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
sh '''
cwd=$(pwd)
rm -rf /tmp/couchjslogs
@@ -212,42 +286,65 @@ pipeline {
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
} // timeout
+ deleteDir()
} // node
},
ubuntu1604erlang183: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 60, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/ubuntu-16.04-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-18.3', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/ubuntu-16.04-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ umask 0
+ rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
+
+ # Build CouchDB packages
+ cd $builddir
+ git clone https://github.com/apache/couchdb-pkg
+ mkdir couchdb
+ cd couchdb
+ cp $cwd/apache-couchdb-*.tar.gz .
+ tar -xf apache-couchdb-*.tar.gz
+ cd ../couchdb-pkg
+ # debian/ubuntu only
+ sudo apt-get install -y libmozjs185-dev
+ platform=$(lsb_release -cs)
+ make $platform PLATFORM=$platform
+ rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
+ # Ubuntu/Debian variant
+ mv ../couchdb/*deb $cwd/pkgs/$platform || true
+
+ # Cleanup
+ cd $cwd && rm -rf $builddir
+ chown -R jenkins:jenkins *
'''
} // withDocker
} // timeout
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ deleteDir()
} // node
},
debian8erlangdefault: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 45, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/debian-8-erlang-default'
- withDockerContainer(image: 'couchdbdev/debian-8-erlang-default', args: '--user 0:0') {
+ withDockerContainer(image: 'couchdbdev/debian-8-erlang-default', args: '-e npm_config_cache=npm-cache -e HOME=. -v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group') {
sh '''
cwd=$(pwd)
rm -rf /tmp/couchjslogs
@@ -258,33 +355,57 @@ pipeline {
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
'''
} // withDocker
} // timeout
+ deleteDir()
} // node
},
debian8erlang183: {
node(label: 'ubuntu') {
- timeout(time: 30, unit: "MINUTES") {
- sh 'rm *.tar.gz || true'
+ timeout(time: 60, unit: "MINUTES") {
unstash 'tarball'
sh 'docker pull couchdbdev/debian-8-erlang-18.3'
- withDockerContainer(image: 'couchdbdev/debian-8-erlang-18.3', args: '--user 0:0') {
+ // must run as root because of sudo step below
+ withDockerContainer(image: 'couchdbdev/debian-8-erlang-18.3', args: '-v=/etc/passwd:/etc/passwd -v /etc/group:/etc/group --user 0:0') {
sh '''
cwd=$(pwd)
- rm -rf /tmp/couchjslogs
- mkdir -p /tmp/couchjslogs
+ rm -rf /tmp/couchjslogs && mkdir -p /tmp/couchjslogs
+
+ # Build CouchDB from tarball
builddir=$(mktemp -d)
cd $builddir
tar -xf $cwd/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make all
- make check || (build-aux/logfile-uploader.py && false)
+ #make check || (build-aux/logfile-uploader.py && false)
+
+ # Build CouchDB packages
+ cd $builddir
+ git clone https://github.com/apache/couchdb-pkg
+ mkdir couchdb
+ cd couchdb
+ cp $cwd/apache-couchdb-*.tar.gz .
+ tar -xf apache-couchdb-*.tar.gz
+ cd ../couchdb-pkg
+ # debian/ubuntu only
+ sudo apt-get install -y libmozjs185-dev
+ platform=$(lsb_release -cs)
+ make $platform PLATFORM=$platform
+ rm -rf $cwd/pkgs/$platform && mkdir -p $cwd/pkgs/$platform
+ # Ubuntu/Debian variant
+ mv ../couchdb/*deb $cwd/pkgs/$platform || true
+
+ # Cleanup
+ cd $cwd && rm -rf $builddir
+ chown -R jenkins:jenkins *
'''
} // withDocker
} // timeout
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ deleteDir()
} // node
}
) // parallel
@@ -293,22 +414,56 @@ pipeline {
stage('Publish') {
when {
- branch '*(master|2.0.x|2.1.x)'
+ expression { return env.BRANCH_NAME ==~ /master|2.0.x|2.1.x|jenkins-.*/ }
}
- agent any
- steps {
- /* Push it somewhere useful other than Jenkins, maybe? */
- /* echo 'Publishing tarball...'
- unstash 'tarball' */
- echo 'Triggering Debian .deb builds...'
- echo 'Triggering Ubuntu .deb builds...'
- echo 'Triggering Ubuntu snap builds...'
- echo 'Triggering CentOS .rpm builds...'
- echo 'Cleaning workspace...'
- sh 'rm -rf * .[a-zA-Z]*'
+ agent {
+ docker {
+ // This image has the deb AND rpm repo tools installed in it
+ image 'couchdbdev/debian-8-base'
+ // We need the jenkins user mapped inside of the image
+ args '-v /etc/passwd:/etc/passwd -v /etc/group:/etc/group'
+ label 'ubuntu'
+ }
}
- }
- }
+ steps {
+ withCredentials([file(credentialsId: 'jenkins-key', variable: 'KEY')]) {
+ sh 'rm -rf pkgs'
+ unarchive mapping: ['pkgs/' : '.']
+ echo 'Building Debian repo...'
+ sh '''
+ git clone https://github.com/apache/couchdb-pkg
+ reprepro -b couchdb-pkg/repo includedeb jessie pkgs/jessie/*deb
+ reprepro -b couchdb-pkg/repo includedeb trusty pkgs/trusty/*deb
+ reprepro -b couchdb-pkg/repo includedeb xenial pkgs/xenial/*deb
+ '''
+ echo 'Building CentOS repos...'
+ sh '''
+ cd pkgs/centos6 && createrepo --database .
+ cd ../centos7 && rm js* && createrepo --database .
+ '''
+ echo 'rsyncing repos to couchdb-vm2...'
+ sh '''
+ mkdir -p $BRANCH_NAME/debian $BRANCH_NAME/el6 $BRANCH_NAME/el7
+ mv couchdb-pkg/repo/pool $BRANCH_NAME/debian
+ mv couchdb-pkg/repo/dists $BRANCH_NAME/debian
+ mv pkgs/centos6/* $BRANCH_NAME/el6
+ mv pkgs/centos7/* $BRANCH_NAME/el7
+ rsync -avz --delete -e "ssh -i $KEY" $BRANCH_NAME username@couchdb-vm2.apache.org:/var/www/html
+ '''
+ // cronjob on couchdb-vm2 cleans up old tarballs (keeps latest 10)
+ echo 'rsyncing source tarball to couchdb-vm2...'
+ unstash 'tarball'
+ sh '''
+ rm -rf $BRANCH_NAME
+ mkdir -p $BRANCH_NAME/source
+ mv apache-couchdb-*.tar.gz $BRANCH_NAME/source
+ rsync -avz -e "ssh -i $KEY" $BRANCH_NAME username@couchdb-vm2.apache.org:/var/www/html
+ '''
+ deleteDir()
+ } // withCredentials
+ } // steps
+ } // stage
+ } // stages
post {
success {
--
To stop receiving notification emails like this one, please contact
"commits@couchdb.apache.org" <co...@couchdb.apache.org>.