You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by basti1302 <gi...@git.apache.org> on 2015/12/22 13:32:12 UTC
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
GitHub user basti1302 opened a pull request:
https://github.com/apache/couchdb-ci/pull/1
Docker container Ubuntu 14.04/Erlang 18.2 for build.a.o
This PR adds a Dockerfile which will produce a container based on Ubuntu 14.04 LTS with Erlang 18.2 and all required dependencies to build current CouchDB master as discussed in [INFRA-10126](https://issues.apache.org/jira/browse/INFRA-10126)).
It also removes a lot of the prototypical stuff that is not needed anymore (based on the discussion in the ticket mentioned above), like provisioning a Jenkins master, configuring build jobs via scm-sync-plug-in etc.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/basti1302/couchdb-ci master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/couchdb-ci/pull/1.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1
----
commit 589e7a8a27446a25db72ce2d87331c87aecfd588
Author: Bastian Krol <ba...@web.de>
Date: 2015-12-21T10:51:39Z
build Docker container that can build CouchDB
- run Ansible inside Docker container
- change Ansible scripts so all required dependencies for building
CouchDB are installed
- remove Jenkins master related scripts
- update readme to reflect new strategy with Docker
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: [wip] Docker container Ubuntu 14.04/Erlan...
Posted by rnewson <gi...@git.apache.org>.
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48264677
--- Diff: readme.markdown ---
@@ -3,44 +3,80 @@ CouchDB CI Setup
Mission statement: Create a new continuous integration infrastructure for the CouchDB project.
-For the background and goals, see this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list.
+For the background and goals, see
-This is the repository for the automated creation of the CouchDB CI infrastructure. Well, at least it will be when it has grown up. This might take a while, though. Right now, it's just a bunch of Ansible scripts, a Vagrantfile and a Veewee definition.
+* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and
+* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).
-See the readme files in folder `baseboxes` (for docs on building the base boxes) and the section about Vagrant below (for docs on how to spin up the setup locally).
+*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
-Fair warning: This is very much work in progress.
+The main purpose of this repository is to provide a number of Docker containers that the ASF infrastructure team can use in their Jenkins setups and which are capable of building CouchDB. The idea is to provide containers for a number of different operating systems and Erlang versions to make sure CouchDB builds and runs on all supported setups.
-Current state:
+The current (rough) plan for the build matrix is this:
-- [x] install bare Jenkins master with Ansible
-- [x] install and configure nginx
-- [x] create CouchDB build job in Jenkins via Ansible
-- [x] switch to master-worker Jenkins setup
-- [x] use ntp server for master and workers
-- [ ] Use SCM sync plug-in to manage job configs
- * http://stackoverflow.com/questions/27138043/jenkins-scm-sync-configuration-plugin-in-docker-wont-talk-to-github
- * https://cburgmer.wordpress.com/2013/01/02/tracking-configuration-changes-in-jenkins/
-- [ ] enable auth for Jenkins
-- [ ] actually fetch CouchDB from VCS
-- [ ] all apt-get commands should pin a specific version, in the base box definition as well as in Ansible. How?
-- [ ] create an additional Ubuntu worker with an older Erlang version
-- [ ] create another base box (different linux distro) for a third worker
-- [ ] talk to Infra people
+**OS/Erlang** | **R14B04** | **R16B03-1** | **17.5** | **18.x**
+--------------------|------------|--------------|----------|---------
+**Ubuntu 14.04** | ? | - | - | WIP
+**Ubuntu latest ?** | ? | - | - | -
+**Debian 7** | ? | - | - | -
+**Debian 8** | ? | - | - | -
+**OS X latest** | ? | - | - | -
+**Free BSD** | ? | - | - | -
+**Windows** | ? | - | - | -
-*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
+### Open questions
+
+* AFAIK Erlang 14 support will be dropped soon-ish, so I'm not sure if it is worth the effort to do anything for that.
+* Which 18.x Erlang version is to be used? I heard someone saying 18.0 once, but that was before 18.1 and 18.2 were available, so I guess it makes more sense to always use the latest 18.x to see if changes in Erlang 18 breaks CouchDB.
--- End diff --
while it's important to test against each OS's stock erlang install (where available), I think it's important to test various erlang versions too, we need to know that our software works on previous and future versions (relative to what's in any stable OS release). Take travis as an example, we can list any (or all) erlang versions there. The main deficiency of travis is that we can't also have an OS axis to that matrix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251592
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
--- End diff --
This requires for tests suite run.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251635
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
+# - erlang-manpages
+# - libnspr4
+# - libnspr4-0d
+# - libnspr4-dev
+# - openssl
--- End diff --
IIRC, openssl requires for Erlang SSL correct work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251579
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
--- End diff --
Dialyzer is need for more complex code inspect which cost significant amount of memory and CPU resources. Would be nice to run it on CI, but it will require some special policy.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by basti1302 <gi...@git.apache.org>.
Github user basti1302 commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251963
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
--- End diff --
Still, a full `make` runs fine without erlang-eunit. I guess the unit tests should definitely run on CI. Do I need to call anything special to trigger them? Or does the Makefile simply skip them if erlang-eunit is not installed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48252688
--- Diff: ansible/roles/ubuntu-worker/tasks/rebar.yml ---
@@ -0,0 +1,37 @@
+- name: create rebar group
+ group:
+ name: rebar
+ state: present
+
+- name: create rebar user
+ user:
+ name: rebar
+ createhome: yes
+ group: rebar
+ state: present
+
+- name: download rebar
+ get_url:
+ url: https://github.com/rebar/rebar/archive/{{ rebar_version }}.tar.gz
--- End diff --
That is need of old past days. Not saying that this is wrong and wont work, but if you can make things simpler - why not?(:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48252154
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
--- End diff --
Just `make` build the sources - it hardly can trigger tests run. `make check` runs all the tests. `make eunit` runs only eunit tests. `make javascript` - only javascript integration test suite. And so on.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: [wip] Docker container Ubuntu 14.04/Erlan...
Posted by rnewson <gi...@git.apache.org>.
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48265648
--- Diff: readme.markdown ---
@@ -3,44 +3,80 @@ CouchDB CI Setup
Mission statement: Create a new continuous integration infrastructure for the CouchDB project.
-For the background and goals, see this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list.
+For the background and goals, see
-This is the repository for the automated creation of the CouchDB CI infrastructure. Well, at least it will be when it has grown up. This might take a while, though. Right now, it's just a bunch of Ansible scripts, a Vagrantfile and a Veewee definition.
+* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and
+* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).
-See the readme files in folder `baseboxes` (for docs on building the base boxes) and the section about Vagrant below (for docs on how to spin up the setup locally).
+*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
-Fair warning: This is very much work in progress.
+The main purpose of this repository is to provide a number of Docker containers that the ASF infrastructure team can use in their Jenkins setups and which are capable of building CouchDB. The idea is to provide containers for a number of different operating systems and Erlang versions to make sure CouchDB builds and runs on all supported setups.
-Current state:
+The current (rough) plan for the build matrix is this:
-- [x] install bare Jenkins master with Ansible
-- [x] install and configure nginx
-- [x] create CouchDB build job in Jenkins via Ansible
-- [x] switch to master-worker Jenkins setup
-- [x] use ntp server for master and workers
-- [ ] Use SCM sync plug-in to manage job configs
- * http://stackoverflow.com/questions/27138043/jenkins-scm-sync-configuration-plugin-in-docker-wont-talk-to-github
- * https://cburgmer.wordpress.com/2013/01/02/tracking-configuration-changes-in-jenkins/
-- [ ] enable auth for Jenkins
-- [ ] actually fetch CouchDB from VCS
-- [ ] all apt-get commands should pin a specific version, in the base box definition as well as in Ansible. How?
-- [ ] create an additional Ubuntu worker with an older Erlang version
-- [ ] create another base box (different linux distro) for a third worker
-- [ ] talk to Infra people
+**OS/Erlang** | **R14B04** | **R16B03-1** | **17.5** | **18.x**
+--------------------|------------|--------------|----------|---------
+**Ubuntu 14.04** | ? | - | - | WIP
+**Ubuntu latest ?** | ? | - | - | -
+**Debian 7** | ? | - | - | -
+**Debian 8** | ? | - | - | -
+**OS X latest** | ? | - | - | -
+**Free BSD** | ? | - | - | -
+**Windows** | ? | - | - | -
-*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
+### Open questions
+
+* AFAIK Erlang 14 support will be dropped soon-ish, so I'm not sure if it is worth the effort to do anything for that.
+* Which 18.x Erlang version is to be used? I heard someone saying 18.0 once, but that was before 18.1 and 18.2 were available, so I guess it makes more sense to always use the latest 18.x to see if changes in Erlang 18 breaks CouchDB.
+* There is no CentOS/RHEL there, shouldn't it be added?
+* Do we run a CouchDB build on all combinations on each commit? This would probably be too much for the ASF Infra build systems. Do we build them once a day? We need to find a good balance between early feedback and resource consumption here.
+* Do we even want to build the master branch or some other branch/tag? I guess the master branch would be most interesting for now, but not entirely sure. Also, it might make sense to make the branch/tag parameterizable so we could also use this to create releases from a specific tag etc.
+* What exactly do we do in each Jenkins build? Just build CouchDB? Also build docs? Start CouchDB? Run some test suite?
--- End diff --
build couchdb, build docs, run all test suites.
The 'build docs' step is very onerous, taking quite a while. If possible, we'd avoid building docs again if they haven't changed since last time, though that might be difficult to achieve.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: [wip] Docker container Ubuntu 14.04/Erlan...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48253451
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
--- End diff --
Actually, `make && make distcheck` (oops, I just realized that we don't have `distcheck`, so `make all check dist`) - that we build and right, all tests passed and we're able to produce ready-to-ship tarball.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: [wip] Docker container Ubuntu 14.04/Erlan...
Posted by rnewson <gi...@git.apache.org>.
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48261940
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
+# - erlang-manpages
+# - libnspr4
+# - libnspr4-0d
+# - libnspr4-dev
+# - openssl
--- End diff --
yes, it does.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by basti1302 <gi...@git.apache.org>.
Github user basti1302 commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48252176
--- Diff: ansible/roles/ubuntu-worker/tasks/rebar.yml ---
@@ -0,0 +1,37 @@
+- name: create rebar group
+ group:
+ name: rebar
+ state: present
+
+- name: create rebar user
+ user:
+ name: rebar
+ createhome: yes
+ group: rebar
+ state: present
+
+- name: download rebar
+ get_url:
+ url: https://github.com/rebar/rebar/archive/{{ rebar_version }}.tar.gz
--- End diff --
Oh, that's good to know. I'll remove the whole rebar stuff. Funny though, both https://github.com/apache/couchdb/blob/master/Dockerfile and https://github.com/klaemo/docker-couchdb/blob/2.0-dev/Dockerfile install rebar separately.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: [wip] Docker container Ubuntu 14.04/Erlan...
Posted by rnewson <gi...@git.apache.org>.
Github user rnewson commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48264818
--- Diff: readme.markdown ---
@@ -3,44 +3,80 @@ CouchDB CI Setup
Mission statement: Create a new continuous integration infrastructure for the CouchDB project.
-For the background and goals, see this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list.
+For the background and goals, see
-This is the repository for the automated creation of the CouchDB CI infrastructure. Well, at least it will be when it has grown up. This might take a while, though. Right now, it's just a bunch of Ansible scripts, a Vagrantfile and a Veewee definition.
+* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and
+* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).
-See the readme files in folder `baseboxes` (for docs on building the base boxes) and the section about Vagrant below (for docs on how to spin up the setup locally).
+*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
-Fair warning: This is very much work in progress.
+The main purpose of this repository is to provide a number of Docker containers that the ASF infrastructure team can use in their Jenkins setups and which are capable of building CouchDB. The idea is to provide containers for a number of different operating systems and Erlang versions to make sure CouchDB builds and runs on all supported setups.
-Current state:
+The current (rough) plan for the build matrix is this:
-- [x] install bare Jenkins master with Ansible
-- [x] install and configure nginx
-- [x] create CouchDB build job in Jenkins via Ansible
-- [x] switch to master-worker Jenkins setup
-- [x] use ntp server for master and workers
-- [ ] Use SCM sync plug-in to manage job configs
- * http://stackoverflow.com/questions/27138043/jenkins-scm-sync-configuration-plugin-in-docker-wont-talk-to-github
- * https://cburgmer.wordpress.com/2013/01/02/tracking-configuration-changes-in-jenkins/
-- [ ] enable auth for Jenkins
-- [ ] actually fetch CouchDB from VCS
-- [ ] all apt-get commands should pin a specific version, in the base box definition as well as in Ansible. How?
-- [ ] create an additional Ubuntu worker with an older Erlang version
-- [ ] create another base box (different linux distro) for a third worker
-- [ ] talk to Infra people
+**OS/Erlang** | **R14B04** | **R16B03-1** | **17.5** | **18.x**
+--------------------|------------|--------------|----------|---------
+**Ubuntu 14.04** | ? | - | - | WIP
+**Ubuntu latest ?** | ? | - | - | -
+**Debian 7** | ? | - | - | -
+**Debian 8** | ? | - | - | -
+**OS X latest** | ? | - | - | -
+**Free BSD** | ? | - | - | -
+**Windows** | ? | - | - | -
-*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
+### Open questions
+
+* AFAIK Erlang 14 support will be dropped soon-ish, so I'm not sure if it is worth the effort to do anything for that.
+* Which 18.x Erlang version is to be used? I heard someone saying 18.0 once, but that was before 18.1 and 18.2 were available, so I guess it makes more sense to always use the latest 18.x to see if changes in Erlang 18 breaks CouchDB.
--- End diff --
so perhaps there's two parts to this 1) confirming couchdb tests run with the standard dependencies from each of the various important OS/distro combinations and 2) confirming couchdb tests run against many versions of important dependencies like erlang itself, even if the OS in those cases remains the same.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by basti1302 <gi...@git.apache.org>.
Github user basti1302 commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48252619
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
+# - erlang-dialyzer
+# - erlang-eunit
--- End diff --
Soo, which steps exactly would you except to run in a regular CI build. `make && make check`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251560
--- Diff: ansible/roles/ubuntu-worker/tasks/dependencies.yml ---
@@ -0,0 +1,59 @@
+- name: import the Erlang Solutions key into apt
+ apt_key:
+ url: http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc
+ state: present
+ # validate_certs: no
+
+- name: add Erlang Solutions deb repository
+ apt_repository:
+ repo: 'deb http://packages.erlang-solutions.com/ubuntu trusty contrib'
+ state: present
+
+# apt-get update
+- name: update apt cache
+ apt:
+ update_cache: yes
+
+- name: install auxiliary packages
+ apt: name={{item}} state=present
+ with_items:
+ - apt-transport-https
+ - curl
+ - git
+ - pkg-config
+ - python
+
+# required for make couch
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - build-essential
+ - ca-certificates
+ - erlang-dev=1:18.2
+ - erlang-nox=1:18.2
+ - erlang=1:18.2
+ - libcurl4-openssl-dev
+ - libicu-dev
+ - libmozjs185-dev
+
+# required for make docs
+- name: install packages required to build CouchDB
+ apt: name={{item}} state=present install_recommends=no
+ with_items:
+ - help2man
+ - python-sphinx
+ - texlive-latex-base
+ - texlive-latex-recommended
+ - texlive-latex-extra
+ - texlive-fonts-recommended
+ - texinfo
+
+# More packages found in klaemo's Dockerfile... not sure if required, apparently not?
+# - erlang-base-hipe
--- End diff --
We can live without hipe, for sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/couchdb-ci/pull/1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251720
--- Diff: ansible/roles/ubuntu-worker/tasks/rebar.yml ---
@@ -0,0 +1,37 @@
+- name: create rebar group
+ group:
+ name: rebar
+ state: present
+
+- name: create rebar user
+ user:
+ name: rebar
+ createhome: yes
+ group: rebar
+ state: present
+
+- name: download rebar
+ get_url:
+ url: https://github.com/rebar/rebar/archive/{{ rebar_version }}.tar.gz
--- End diff --
No need in rebar: ./configure script [takes care](https://github.com/apache/couchdb/blob/master/configure#L652-L667) about it. While you stay with `make *` commands, it will be used and right version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251847
--- Diff: readme.markdown ---
@@ -3,44 +3,80 @@ CouchDB CI Setup
Mission statement: Create a new continuous integration infrastructure for the CouchDB project.
-For the background and goals, see this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list.
+For the background and goals, see
-This is the repository for the automated creation of the CouchDB CI infrastructure. Well, at least it will be when it has grown up. This might take a while, though. Right now, it's just a bunch of Ansible scripts, a Vagrantfile and a Veewee definition.
+* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and
+* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).
-See the readme files in folder `baseboxes` (for docs on building the base boxes) and the section about Vagrant below (for docs on how to spin up the setup locally).
+*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
-Fair warning: This is very much work in progress.
+The main purpose of this repository is to provide a number of Docker containers that the ASF infrastructure team can use in their Jenkins setups and which are capable of building CouchDB. The idea is to provide containers for a number of different operating systems and Erlang versions to make sure CouchDB builds and runs on all supported setups.
-Current state:
+The current (rough) plan for the build matrix is this:
-- [x] install bare Jenkins master with Ansible
-- [x] install and configure nginx
-- [x] create CouchDB build job in Jenkins via Ansible
-- [x] switch to master-worker Jenkins setup
-- [x] use ntp server for master and workers
-- [ ] Use SCM sync plug-in to manage job configs
- * http://stackoverflow.com/questions/27138043/jenkins-scm-sync-configuration-plugin-in-docker-wont-talk-to-github
- * https://cburgmer.wordpress.com/2013/01/02/tracking-configuration-changes-in-jenkins/
-- [ ] enable auth for Jenkins
-- [ ] actually fetch CouchDB from VCS
-- [ ] all apt-get commands should pin a specific version, in the base box definition as well as in Ansible. How?
-- [ ] create an additional Ubuntu worker with an older Erlang version
-- [ ] create another base box (different linux distro) for a third worker
-- [ ] talk to Infra people
+**OS/Erlang** | **R14B04** | **R16B03-1** | **17.5** | **18.x**
+--------------------|------------|--------------|----------|---------
+**Ubuntu 14.04** | ? | - | - | WIP
+**Ubuntu latest ?** | ? | - | - | -
+**Debian 7** | ? | - | - | -
+**Debian 8** | ? | - | - | -
+**OS X latest** | ? | - | - | -
+**Free BSD** | ? | - | - | -
+**Windows** | ? | - | - | -
-*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
+### Open questions
+
+* AFAIK Erlang 14 support will be dropped soon-ish, so I'm not sure if it is worth the effort to do anything for that.
+* Which 18.x Erlang version is to be used? I heard someone saying 18.0 once, but that was before 18.1 and 18.2 were available, so I guess it makes more sense to always use the latest 18.x to see if changes in Erlang 18 breaks CouchDB.
--- End diff --
Imho, need to ensure that we run correctly with the latest and system default if it matches the requirement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-ci pull request: Docker container Ubuntu 14.04/Erlang 18.2...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on a diff in the pull request:
https://github.com/apache/couchdb-ci/pull/1#discussion_r48251780
--- Diff: readme.markdown ---
@@ -3,44 +3,80 @@ CouchDB CI Setup
Mission statement: Create a new continuous integration infrastructure for the CouchDB project.
-For the background and goals, see this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list.
+For the background and goals, see
-This is the repository for the automated creation of the CouchDB CI infrastructure. Well, at least it will be when it has grown up. This might take a while, though. Right now, it's just a bunch of Ansible scripts, a Vagrantfile and a Veewee definition.
+* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and
+* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).
-See the readme files in folder `baseboxes` (for docs on building the base boxes) and the section about Vagrant below (for docs on how to spin up the setup locally).
+*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
-Fair warning: This is very much work in progress.
+The main purpose of this repository is to provide a number of Docker containers that the ASF infrastructure team can use in their Jenkins setups and which are capable of building CouchDB. The idea is to provide containers for a number of different operating systems and Erlang versions to make sure CouchDB builds and runs on all supported setups.
-Current state:
+The current (rough) plan for the build matrix is this:
-- [x] install bare Jenkins master with Ansible
-- [x] install and configure nginx
-- [x] create CouchDB build job in Jenkins via Ansible
-- [x] switch to master-worker Jenkins setup
-- [x] use ntp server for master and workers
-- [ ] Use SCM sync plug-in to manage job configs
- * http://stackoverflow.com/questions/27138043/jenkins-scm-sync-configuration-plugin-in-docker-wont-talk-to-github
- * https://cburgmer.wordpress.com/2013/01/02/tracking-configuration-changes-in-jenkins/
-- [ ] enable auth for Jenkins
-- [ ] actually fetch CouchDB from VCS
-- [ ] all apt-get commands should pin a specific version, in the base box definition as well as in Ansible. How?
-- [ ] create an additional Ubuntu worker with an older Erlang version
-- [ ] create another base box (different linux distro) for a third worker
-- [ ] talk to Infra people
+**OS/Erlang** | **R14B04** | **R16B03-1** | **17.5** | **18.x**
+--------------------|------------|--------------|----------|---------
+**Ubuntu 14.04** | ? | - | - | WIP
+**Ubuntu latest ?** | ? | - | - | -
+**Debian 7** | ? | - | - | -
+**Debian 8** | ? | - | - | -
+**OS X latest** | ? | - | - | -
+**Free BSD** | ? | - | - | -
+**Windows** | ? | - | - | -
-*Remark: Throughout this repository we use the terms "master"/"worker" for the Jenkins build machines, whereas the Jenkins documentation uses the terms "master"/"slave".*
+### Open questions
+
+* AFAIK Erlang 14 support will be dropped soon-ish, so I'm not sure if it is worth the effort to do anything for that.
--- End diff --
Erlang R14 support is already dropped with no return. So R16B03-1+.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---