You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@couchdb.apache.org by ko...@apache.org on 2019/08/07 01:23:54 UTC
[couchdb] branch jenkins-junit-reports-and-jenkins-env updated
(cbb09ae -> 0533bb2)
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a change to branch jenkins-junit-reports-and-jenkins-env
in repository https://gitbox.apache.org/repos/asf/couchdb.git.
discard cbb09ae Switch to sequential stages everywhere
discard 8ada0e9 Fix the arm platform label
discard 753af88 Run the full suite
discard b361618 Reactivate BSD tests
discard f586087 Add timeout for couch_db_split_tests
discard bae8c2d Skip flaky test for now
discard 80251c2 Fix bash-ism in EUnit retry logic
discard b7b772d Revert "Try suggested fix for retry loop"
discard 0872824 Merge remote-tracking branch 'gitbox/master' into junit-reports-and-jenkins-env
discard 307fa41 Try suggested fix for retry loop
discard d152076 Few more experiments
discard 7ffd407 Forgot to validate
discard 0de3324 Few more tweaks
discard 7b5cf08 Remove archive after extraction to avoid confusing cd
discard 6b9cd14 Use git plugin
discard 59e4213 Reenable all platforms
discard 3c26a49 Plugin not installed
discard 10633ea Drop builddir
discard 457430f Can't resolve env vars here
discard 714270b Quoting issue?
discard 99b5519 Define customWorkspace in docker block
discard 528fba0 Play with sequential stages
discard a12ef83 More syntax games
discard 64b6c10 Fix braces error
discard 94a11b4 Use rsync to bring reports back to workspace
discard 86e00e4 Try using dir()
discard 0a2e637 Always return to the builddir
discard 6f548e7 Don't blow away what we just unstashed
discard 1b0f2eb Env vars are case-sensitive :)
discard 0fe8827 WIP
discard d78a640 Clean build dir first, do a full build before eunit
discard 4bc1498 See if the problem is unique to freebsd
discard 2d0d757 Run the full suite on freebsd
discard 203fa63 Use environment directly instead of withEnv
discard 3f3f40e Try getting tmp dir from Jenkinsfile
discard 6d85920 Picked the wrong app to isolate
discard 8914a43 Temporarily shorten the test suite even further
discard 7e432c8 Try without builddir
discard 8936476 Getting sloppy
discard 5803ef5 Look in the proper source extract for test reports
discard 53256cd Try eunit instead of elixir
discard 4600379 Oopsie
discard defe061 Merge remote-tracking branch 'gitbox/master' into junit-reports-and-jenkins-env
discard 0512f6b Defer cleanup to post so we can grab test results
discard e96f528 And more
discard 5937147 More debugging
omit 1d898a8 Run all builds to completion temporarily
omit 7bae5b6 Fix paths on FreeBSD, reorder mv commands
omit 7a890d2 Merge branch 'master' into junit-reports-and-jenkins-env
omit 7194125 Save XML reports for EUnit tests as well
omit d50c8ab Publish ExUnit test results up to Jenkins ExUnit
omit bce143c Restore JUnitFormatter
omit c17cc94 Set MIX_HOME on ARM builds
add d5ae1bf Fix bash-ism in EUnit retry logic
new bec80ee Fix copy/paste errors in platform naming
new 9bc9d22 Refactor using sequential stages, in workspace
new d59ab94 Capture EUnit and ExUnit test results for Jenkins
new 0533bb2 Add timeout for couch_db_split_tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (cbb09ae)
\
N -- N -- N refs/heads/jenkins-junit-reports-and-jenkins-env (0533bb2)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
Jenkinsfile | 102 ++++++-------
.../src/cpse_test_purge_replication.erl | 164 ++++++++++-----------
2 files changed, 123 insertions(+), 143 deletions(-)
[couchdb] 03/04: Capture EUnit and ExUnit test results for Jenkins
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch jenkins-junit-reports-and-jenkins-env
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit d59ab942e79a2229e298fdf857ad2acb5f23fd91
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Tue Aug 6 21:12:18 2019 -0400
Capture EUnit and ExUnit test results for Jenkins
---
Jenkinsfile | 38 ++++++++++++++++++++++++++++++++++++++
mix.exs | 1 +
rebar.config.script | 2 +-
3 files changed, 40 insertions(+), 1 deletion(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index b2deb8e..f37e9bf 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -144,6 +144,9 @@ pipeline {
} // withEnv
} // steps
post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
cleanup {
sh 'rm -rf $COUCHDB_IO_LOG_DIR'
}
@@ -173,6 +176,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -215,6 +223,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -258,6 +271,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -300,6 +318,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -342,6 +365,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -384,6 +412,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
@@ -426,6 +459,11 @@ pipeline {
unstash 'tarball'
sh( script: build_and_test )
}
+ post {
+ always {
+ junit '**/.eunit/*.xml, **/_build/*/lib/couchdbtest/*.xml'
+ }
+ }
}
stage('Build CouchDB packages') {
steps {
diff --git a/mix.exs b/mix.exs
index 2e213ae..43c8b6d 100644
--- a/mix.exs
+++ b/mix.exs
@@ -35,6 +35,7 @@ defmodule CouchDBTest.Mixfile do
# Run "mix help deps" to learn about dependencies.
defp deps() do
[
+ {:junit_formatter, "~> 3.0", only: [:dev, :test, :integration]},
{:httpotion, "~> 3.0", only: [:dev, :test, :integration], runtime: false},
{:jiffy, path: Path.expand("src/jiffy", __DIR__)},
{:ibrowse,
diff --git a/rebar.config.script b/rebar.config.script
index 6445057..c38b6e2 100644
--- a/rebar.config.script
+++ b/rebar.config.script
@@ -156,7 +156,7 @@ AddConfig = [
{sub_dirs, SubDirs},
{lib_dirs, ["src"]},
{erl_opts, [{i, "../"} | ErlOpts]},
- {eunit_opts, [verbose]},
+ {eunit_opts, [verbose, {report,{eunit_surefire,[{dir,"."}]}}]},
{plugins, [eunit_plugin]},
{dialyzer, [
{plt_location, local},
[couchdb] 01/04: Fix copy/paste errors in platform naming
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch jenkins-junit-reports-and-jenkins-env
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit bec80ee4b878472ee32a9344da482cc7b08fb03c
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Tue Aug 6 16:48:24 2019 -0400
Fix copy/paste errors in platform naming
---
Jenkinsfile | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index c575748..3c49b76 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -246,7 +246,7 @@ pipeline {
timeout(time: 90, unit: "MINUTES")
}
environment {
- platform = 'xenial'
+ platform = 'bionic'
}
steps {
sh 'rm -f apache-couchdb-*.tar.gz'
@@ -300,7 +300,7 @@ pipeline {
timeout(time: 90, unit: "MINUTES")
}
environment {
- platform = 'jessie'
+ platform = 'stretch'
}
steps {
sh 'rm -f apache-couchdb-*.tar.gz'
@@ -327,7 +327,7 @@ pipeline {
timeout(time: 90, unit: "MINUTES")
}
environment {
- platform = 'jessie'
+ platform = 'aarch64-debian-stretch'
}
steps {
sh 'rm -f apache-couchdb-*.tar.gz'
[couchdb] 04/04: Add timeout for couch_db_split_tests
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch jenkins-junit-reports-and-jenkins-env
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 0533bb25c0d9bbe4c75e0f38e0af1ca0b993ee3a
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Tue Aug 6 13:28:47 2019 -0400
Add timeout for couch_db_split_tests
The "Should copy local docs after split in four" test was occasionally
timing out in CI.
---
src/couch/test/eunit/couch_db_split_tests.erl | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/couch/test/eunit/couch_db_split_tests.erl b/src/couch/test/eunit/couch_db_split_tests.erl
index 7d2bb40..c44f47c 100644
--- a/src/couch/test/eunit/couch_db_split_tests.erl
+++ b/src/couch/test/eunit/couch_db_split_tests.erl
@@ -16,6 +16,7 @@
-include_lib("couch/include/couch_db.hrl").
-define(RINGTOP, 2 bsl 31).
+-define(TIMEOUT, 60). % seconds
setup() ->
@@ -68,7 +69,7 @@ should_split_shard({Desc, TotalDocs, Q}, DbName) ->
TMap = make_targets(Ranges),
DocsPerRange = TotalDocs div Q,
PickFun = make_pickfun(DocsPerRange),
- {Desc, ?_test(begin
+ {Desc, timeout, ?TIMEOUT, ?_test(begin
{ok, UpdateSeq} = couch_db_split:split(DbName, TMap, PickFun),
?assertEqual(ExpectSeq, UpdateSeq),
maps:map(fun(Range, Name) ->
[couchdb] 02/04: Refactor using sequential stages, in workspace
Posted by ko...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kocolosk pushed a commit to branch jenkins-junit-reports-and-jenkins-env
in repository https://gitbox.apache.org/repos/asf/couchdb.git
commit 9bc9d22f87f6bd9d044f859f888175358427bddc
Author: Adam Kocoloski <ko...@apache.org>
AuthorDate: Tue Aug 6 16:49:52 2019 -0400
Refactor using sequential stages, in workspace
This work moves the builds back into the workspace, using a separate
sub-directory per platform to avoid clashes between builds caused by
JENKINS-57454. It also breaks out the steps into a pair of sequential
stages within each each parallel stage of the build, which gives us
better visibility into the progress of the build, and also sets us up
to capture test results and expose them directly via Jenkins UI for
faster problem determination.
---
Jenkinsfile | 249 ++++++++++++++++++++++++++++++++++++++++++------------------
1 file changed, 177 insertions(+), 72 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 3c49b76..b2deb8e 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -14,35 +14,33 @@
// the License.
// DRYing out the Jenkinsfile...
-build_script = '''
-mkdir -p ${COUCHDB_IO_LOG_DIR}
-echo
-echo "Build CouchDB from tarball & test"
-builddir=$(mktemp -d)
-cd ${builddir}
+build_and_test = '''
+mkdir -p ${COUCHDB_IO_LOG_DIR}
+rm -rf build
+mkdir build
+cd build
tar -xf ${WORKSPACE}/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
make check || (build-aux/logfile-uploader.py && false)
+'''
-echo
-echo "Build CouchDB packages"
-cd ${builddir}
+make_packages = '''
git clone https://github.com/apache/couchdb-pkg
+rm -rf couchdb
mkdir couchdb
cp ${WORKSPACE}/apache-couchdb-*.tar.gz couchdb
tar -xf ${WORKSPACE}/apache-couchdb-*.tar.gz -C couchdb
cd couchdb-pkg
make ${platform} PLATFORM=${platform}
+'''
-echo
-echo "Cleanup & save for posterity"
+cleanup_and_save = '''
rm -rf ${WORKSPACE}/pkgs/${platform}
mkdir -p ${WORKSPACE}/pkgs/${platform}
-mv ../rpmbuild/RPMS/$(arch)/*rpm ${WORKSPACE}/pkgs/${platform} || true
-mv ../couchdb/*.deb ${WORKSPACE}/pkgs/${platform} || true
-rm -rf ${builddir} ${COUCHDB_IO_LOG_DIR}
+mv ${WORKSPACE}/rpmbuild/RPMS/$(arch)/*rpm ${WORKSPACE}/pkgs/${platform} || true
+mv ${WORKSPACE}/couchdb/*.deb ${WORKSPACE}/pkgs/${platform} || true
'''
pipeline {
@@ -113,11 +111,7 @@ pipeline {
// https://issues.jenkins-ci.org/browse/JENKINS-47962
// https://issues.jenkins-ci.org/browse/JENKINS-48050
- // The builddir stuff is to prevent all the builds from live syncing
- // their build results to each other during the build, which ACTUALLY
- // HAPPENS. Ugh.
-
- stage('make check') {
+ stage('Test and Package') {
parallel {
@@ -138,18 +132,22 @@ pipeline {
mkdir -p $COUCHDB_IO_LOG_DIR
# Build CouchDB from tarball & test
- builddir=$(mktemp -d)
- cd $builddir
+ mkdir build
+ cd build
tar -xf $WORKSPACE/apache-couchdb-*.tar.gz
cd apache-couchdb-*
./configure --with-curl
gmake check || (build-aux/logfile-uploader.py && false)
# No package build for FreeBSD at this time
- rm -rf $builddir $COUCHDB_IO_LOG_DIR
'''
} // withEnv
} // steps
+ post {
+ cleanup {
+ sh 'rm -rf $COUCHDB_IO_LOG_DIR'
+ }
+ } // post
} // stage FreeBSD
stage('CentOS 6') {
@@ -158,6 +156,8 @@ pipeline {
image 'couchdbdev/centos-6-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ // this keeps builds landing on the same host from clashing with each other
+ customWorkspace pwd() + '/centos6'
}
}
options {
@@ -167,14 +167,28 @@ pipeline {
environment {
platform = 'centos6'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -185,6 +199,7 @@ pipeline {
image 'couchdbdev/centos-7-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ customWorkspace pwd() + '/centos7'
}
}
options {
@@ -194,14 +209,29 @@ pipeline {
environment {
platform = 'centos7'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ unstash 'tarball'
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -212,6 +242,7 @@ pipeline {
image 'couchdbdev/ubuntu-xenial-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ customWorkspace pwd() + '/xenial'
}
}
options {
@@ -221,14 +252,28 @@ pipeline {
environment {
platform = 'xenial'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -239,6 +284,7 @@ pipeline {
image 'couchdbdev/ubuntu-bionic-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ customWorkspace pwd() + '/bionic'
}
}
options {
@@ -248,14 +294,28 @@ pipeline {
environment {
platform = 'bionic'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -266,6 +326,7 @@ pipeline {
image 'couchdbdev/debian-jessie-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ customWorkspace pwd() + '/jessie'
}
}
options {
@@ -275,14 +336,28 @@ pipeline {
environment {
platform = 'jessie'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -293,6 +368,7 @@ pipeline {
image 'couchdbdev/debian-stretch-erlang-19.3.6:latest'
alwaysPull true
label 'ubuntu'
+ customWorkspace pwd() + '/stretch'
}
}
options {
@@ -302,14 +378,28 @@ pipeline {
environment {
platform = 'stretch'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -320,6 +410,7 @@ pipeline {
image 'couchdbdev/aarch64-debian-stretch-erlang-20.3.8.20:latest'
alwaysPull true
label 'arm'
+ customWorkspace pwd() + '/arm'
}
}
options {
@@ -329,14 +420,28 @@ pipeline {
environment {
platform = 'aarch64-debian-stretch'
}
- steps {
- sh 'rm -f apache-couchdb-*.tar.gz'
- unstash 'tarball'
- sh( script: build_script )
- } // steps
+ stages {
+ stage('Build from tarball & test') {
+ steps {
+ unstash 'tarball'
+ sh( script: build_and_test )
+ }
+ }
+ stage('Build CouchDB packages') {
+ steps {
+ sh( script: make_packages )
+ sh( script: cleanup_and_save )
+ }
+ post {
+ success {
+ archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ }
+ }
+ }
+ } // stages
post {
- success {
- archiveArtifacts artifacts: 'pkgs/**', fingerprint: true
+ cleanup {
+ sh 'rm -rf ${WORKSPACE}/*'
}
} // post
} // stage
@@ -442,7 +547,7 @@ pipeline {
body: "Boo, we failed. ${env.RUN_DISPLAY_URL}"
}
cleanup {
- sh 'rm -rf ${WORKSPACE}/*'
+ sh 'rm -rf ${COUCHDB_IO_LOG_DIR}'
}
}