[jira] [Comment Edited] (INFRA-10126) CI infrastructure for CouchDB

["Bastian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/INFRA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706534#comment-14706534 ] 

Bastian edited comment on INFRA-10126 at 8/21/15 12:11 PM:
-----------------------------------------------------------

I totally understand that you do not want a separate master. I was only bringing that up for one reason: I guess that in the end there will be a lot of CouchDB jobs. Right now we are planning for 7 to 8 OSes and up to 3 to 4 Erlang versions. Building different versions of CouchDB might or might not come into play later.

OSes:
- Ubuntu 14 LTS
- Ubuntu 15 LTS
- Debian 7
- Debian 8
- OS X latest
- Free BSD
- Windows

Erlang versions: 
- R14B04
- R16B03-1
- 17.5
- 18.0

So we are talking about ~30 jobs. (The matrix might be a bit smaller, not all OSes have all Erlang versions). My main concern here is that it might not be feasible to maintain these jobs manually. That's why I am trying to put as much of the configuration and setup into code.

How do you currently maintain job configurations? Is it just each project that maintains their jobs manually? Or do you use something like the scm-sync-plug-in or the job-dsl-plugin? If not, would you be open to adding something like that to Jenkins?

In a similar fashion, how are slave machines maintained? I see you already have quite a number of slaves for different OSes, that's great. To build and test CouchDB, we would need put all of CouchDB dependencies on these machines. Again, preferably in an automated, non-manual fashion. Since you say the pool of slaves is limited, it might be a good alternative to have separate build slaves for CouchDB that are not from that pool and that we can set up ad lib (assuming we can find someone to provide the resources).

Kind regards

  Bastian


was (Author: bastian.krol):
I totally understand that you do not want a separate master. I was only bringing that up for one reason: I guess that in the end there will be a lot of CouchDB jobs. Right now we are planning for 7 to 8 OSes and up to 3 to 4 Erlang versions. Building different versions of CouchDB might or might not come into play later.

OSes:
- Ubuntu 14 LTS
- Ubuntu 15 LTS
- Debian 7
- Debian 8
- OS X latest
- Free BSD
- Windows

Erlang versions: 
- R14B04
- R16B03-1
- 17.5
- 18.0

So we are talking about ~30 jobs. My main concern here is that it might not be feasible to maintain these jobs manually. That's why I am trying to put as much of the configuration and setup into code.

How do you currently maintain job configurations? Is it just each project that maintains their jobs manually? Or do you use something like the scm-sync-plug-in or the job-dsl-plugin? If not, would you be open to adding something like that to Jenkins?

In a similar fashion, how are slave machines maintained? I see you already have quite a number of slaves for different OSes, that's great. To build and test CouchDB, we would need put all of CouchDB dependencies on these machines. Again, preferably in an automated, non-manual fashion. Since you say the pool of slaves is limited, it might be a good alternative to have separate build slaves for CouchDB that are not from that pool and that we can set up ad lib (assuming we can find someone to provide the resources).

Kind regards

  Bastian

> CI infrastructure for CouchDB
> -----------------------------
>
>                 Key: INFRA-10126
>                 URL: https://issues.apache.org/jira/browse/INFRA-10126
>             Project: Infrastructure
>          Issue Type: Task
>          Components: Jenkins
>            Reporter: Bastian
>            Assignee: Andrew Bayer
>
> The CouchDB project wants to up their CI game. Right now there is only an undermaintained Jenkins installation on a box in Jan Lehnhardt's flat. The goal is to build and test CouchDB on as many different OSes and Erlang versions as possible. In the end, this will probably a matrix of build slaves. I volunteered to help with that/take care of that. I already started to create an Ansible project that provisions a Jenkins server and several slaves for different OSes.
> But in the end this will probably need to run on some "official" ASF CI infrastructure.
> The two options I see are
> 1) Integrate this into the existing Jenkins infrastructure?
> 2) Run it on different VMs provided by the ASF infra team?
> I read some of the docs at https://cwiki.apache.org/confluence/display/REEF/Continuous+Integration+Setup and saw that the usual way is to maintain the job configs manually. This might be quite a lot of effort since we will need a large number of jobs (for the matrix of configurations). We plan to automate as much as possible, even the job configs and the slave configs etc. Of course, this might conflict with the existing Jenkins setup.
> Which of the two options is viable? What is the best way to proceed here?
> Kind regards
>   Bastian



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)