[jira] [Commented] (BIGTOP-1323) provide an option to containerize Bigtop package build

["Julien Eid (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054340#comment-14054340 ] 

Julien Eid commented on BIGTOP-1323:
------------------------------------

Hi Roman,

I’m really glad to see this ticket come up, I’ve been doing some work on getting Jenkins slave containerized builds going as well as building Bigtop in containers so I was very excited to see this! I totally agree with everything you said about our current Jenkins setup and I’ve seen the hell that can come from having a matrix of build platforms and slaves that are all configured a different way and are very brittle. I have a bunch of recommendations on how we can fix this issue and get our build process cleaned up and working automagically without human intervention while making our build environment on Jenkins match the build environment on people’s personal computers.

I have made a separate JIRA issue to talk about how to integrate our Jenkins setup with Docker containers. I separated the integration of the Jenkins side of things from this ticket as it appeared to me to be a separate but related issue to having the ability to have builds be done in containers. The link can be found here: https://issues.apache.org/jira/browse/BIGTOP-1368

As for just doing builds in containers, I have done a fair chunk of work on getting BigTop builds happening in Docker containers. I have an experimental git repo with some Dockerfiles that are a pretty good prototype of doing Bigtop builds in containers as well having some prototype Dockerfiles for my seperate Jenkins ticket. It’s located at https://github.com/jeid64/bigtop-dockerfiles The build/ folder has a working Ubuntu 12.04 Dockerfile that will use the Puppet modules for the toolchain to configure the container to do builds later on. This Dockerfile works as far as I can see, I’ve built both HBase and Hive in a container. We can use this Docker image to build on Jenkins as well as on dev’s own computers. I’ve built the image and pushed it to the public Docker hub. You can use my Docker image located here with the following command: 

docker run -i -t jeid64/build-precise bash -c 'source /etc/profile.d/bigtop.sh; cd /bigtop ; make hbase-deb;'

It currently does not put the artifacts anywhere, but that can be handled easily by a mount volume that gets the resultant packages and is accessible by the host.

That Dockerfile for 12.04 can be quickly copied and modified for different versions of Ubuntu. I can make Dockerfiles for our other platforms easily if they are configurable from Puppet for the toolchain. I saw with Bigtop-1359 there was some work to add more platforms and I think that’s the best way to have the ability to easily make containers with the toolchain. I’ve already created a Centos 6.4 image and will work on supporting all of our platforms.

I would love to be assigned to this ticket along with anyone else that wants to work on it and get everything setup.

> provide an option to containerize Bigtop package build 
> -------------------------------------------------------
>
>                 Key: BIGTOP-1323
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1323
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: Build
>    Affects Versions: 0.8.0
>            Reporter: Roman Shaposhnik
>            Assignee: Roman Shaposhnik
>             Fix For: 0.8.0
>
>
> Looking at the state of our Jenkins made me scream "I'm mad as Hell and I'm not going to take this anymore". The basic problem is simple: our build slaves are not actually managed by Puppet (even though we do have Puppet code to do that) and they seem to bitrot quite a bit (e.g. git clone from ASF repos on SLES and CentOS5 is totally borked).
> There's another, somewhat subtler, problem: currently we have to keep as many slaves as we've got platforms to support. This leads to a pretty poor utilization and if a slave goes down requires manual intervention.
> What I'd like to introduce in this JIRA is a notion of decoupling OS environment dependency from the base OS. The technology I'm proposing here is Docker (which really is just a prettier UI on top of Linux Containers). IOW, regardless of what OS you'd be running on your host -- as long as you've got docker you can have an exact replica of a target os running in a container.
> Here's my plan for simplifying the management aspect of what's going on. Note, that this is a complimentary plan. Whatever is happening (or not happening!) today on our Jenkins can remain there for as long as we need it.
>   # make our build slaves be 100% uniform and fungible instances of CoreOS VMs https://coreos.com/ The benefit of CoreOS is pretty much 0 administration cost, quick boot up time, full integration with Docker (see bellow) and full integration with things like VirtualBox, etc (so that you can replicate exactly the kind of process that we will setup on the Jenkins).
>   # make the actual builds happen in a container using CoreOS's integration with Docker. IOW, instead of running: make foo on the slave the actual command will look like: docker run -i -t -v `pwd`:/workspace:rw BUILD-ENV bash -c 'cd /workspace ; make foo'
>   # add jobs to the jenkins that would maintain images of BUILD-ENV containers. Each image will be a base-line OS (RHEL, Ubuntu, etc) plus all the build dependencies of a given source package (but nothing else!).
> All of these 3 steps are pretty easy to automate and they do have an added benefit of being 100% replicatable on your workstation. IOW, as long as you have docker running on your host OS you can build packages for any OS with a simple command.
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.2#6252)