[jira] [Created] (BIGTOP-1323) provide an option to containerize Bigtop package build

["Roman Shaposhnik (JIRA)" <ji...@apache.org>]

Roman Shaposhnik created BIGTOP-1323:
----------------------------------------

             Summary: provide an option to containerize Bigtop package build 
                 Key: BIGTOP-1323
                 URL: https://issues.apache.org/jira/browse/BIGTOP-1323
             Project: Bigtop
          Issue Type: Improvement
          Components: Build
    Affects Versions: 0.8.0
            Reporter: Roman Shaposhnik
            Assignee: Roman Shaposhnik
             Fix For: 0.8.0


Looking at the state of our Jenkins made me scream "I'm mad as Hell and I'm not going to take this anymore". The basic problem is simple: our build slaves are not actually managed by Puppet (even though we do have Puppet code to do that) and they seem to bitrot quite a bit (e.g. git clone from ASF repos on SLES and CentOS5 is totally borked).

There's another, somewhat subtler, problem: currently we have to keep as many slaves as we've got platforms to support. This leads to a pretty poor utilization and if a slave goes dow requires manual intervention.

What I'd like to introduce in this JIRA is a notion of decoupling OS environment dependency from the base OS. The technology I'm proposing here is Docker (which really is just a prettier UI on top of Linux Containers). IOW, regardless of what OS you'd be running on your host -- as long as you've got docker you can have an exact replica of a target os running in a container.

Here's my plan for simplifying the management aspect of what's going on. Note, that this is a complimentary plan. Whatever is happening (or not happening!) today on our Jenkins can remain there for as long as we need it.
  # make our build slaves be 100% uniform and fungible instances of CoreOS VMs https://coreos.com/ The benefit of CoreOS is pretty much 0 administration cost, quick boot up time, full integration with Docker (see bellow) and full integration with things like VirtualBox, etc (so that you can replicate exactly the kind of process that we will setup on the Jenkins).
  # make the actual builds happen in a container using CoreOS's integration with Docker. IOW, instead of running: make foo on the salve the actual command will look like: docker run -i -t -v `pwd`:/workspace:rw BUILD-ENV bash -c 'cd /workspace ; make foo'
  # add jobs to the jenkins that would maintain images of BUILD-ENV containers. Each image will be a base-line OS (RHEL, Ubuntu, etc) plus all the build dependencies of a given source package (but nothing else!).

All of these 3 steps are pretty easy to automate and they do have an added benefit of being 100% replicatable on your workstation. IOW, as long as you have docker running on your host OS you can build packages for any OS with a simple command.

Thoughts?



--
This message was sent by Atlassian JIRA
(v6.2#6252)