[jira] [Commented] (BIGTOP-720) Build environment automation

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13623368#comment-13623368 ] 

Konstantin Boudnik commented on BIGTOP-720:
-------------------------------------------

Great start, Ian. A couple of comments:
- the patch delivers the recipes to {{bigtop-toolchain/puppet}} while classes definition expect module name to be {{puppet-bigtoptc}}. This renders the patch unusable unless {{% mv puppet puppet-bigtoptc}} first
- I would suggest to call the module just {{bigtoptc}}, perhaps, or just {{bigtop-toolchain}} so it can be used directly from the toplevel directory of the Bigtop project
- class {{forrest}} can be simplified to
{noformat}
class puppet-bigtoptc::forrest{

  file{ '/tmp/apache-forrest-0.9.tar.gz':
    source => 'puppet:///modules/puppet-bigtoptc/apache-forrest-0.9.tar.gz',
    ensure => present,
    owner  => root,
    group  => root,
    mode   => 755
  }

  exec{'/bin/tar xvzf /tmp/apache-forrest-0.9.tar.gz':
    cwd         => '/usr/local',
    refreshonly => true,
    subscribe   => File["/tmp/apache-forrest-0.9.tar.gz"],
  }

  file { '/usr/local/apache-forrest':
    ensure  => link,
    target  => '/usr/local/apache-forrest-0.9',
    require => Exec['/bin/tar xvzf /tmp/apache-forrest-0.9.tar.gz'],
  }
}
{noformat}
if one use complete binary distribution from [here|http://archive.apache.org/dist/forrest/0.9/apache-forrest-0.9.tar.gz}}
- I'd suggest to ensure the presence of needed tarballs/packages in {{files/}} and download them if needed using the same puppet recipes, instead of stating the requirement for a user to download them first. The whole purpose of this exercise is to provide a 'single button push' dev. environment creation. I think this can be done as a separate JIRA; for now we can just update README file with the links to the tarballs. Something like:
{noformat}
The following files must also be downloaded from their respective mirrors and copied into files/

apache-ant-1.9.0-bin.tar.gz
  http://mirrors.ibiblio.org/apache//ant/binaries/apache-ant-1.9.0-bin.tar.gz

apache-forrest-0.9-dependencies.tar.gz
  http://archive.apache.org/dist/forrest/0.9/apache-forrest-0.9.tar.gz

apache-forrest-0.9-sources.tar.gz

apache-maven-3.0.5-bin.tar.gz
  ftp://mirror.reverse.net/pub/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz
{noformat}
- class {{jdk}} won't work on Ubuntu: it needs to have ability to install either rpm or deb package. This is one can be resolved later as a separate JIRA. 
                
> Build environment automation
> ----------------------------
>
>                 Key: BIGTOP-720
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-720
>             Project: Bigtop
>          Issue Type: Task
>          Components: General
>            Reporter: Anatoli Fomenko
>            Assignee: Anatoli Fomenko
>         Attachments: 0001-Add-bigtop-toolchain-and-puppet-manifests-for-deploy.patch
>
>
> Summary of the bigtop-dev discussion:
> A few approaches have been suggested:
> # (Roman) maintain a parallel (very shallow) collection of puppet code that would, essentially, manage our "build slaves"
> # (Roman) do #1 but automate it in such a way that the info actually gets harvested from spec/conrol files. (Cos suggested to use Groovy for implementation)
> # (Bruno) VMs.
> There are some pros and cons for each of them.
> For example:
> #3 Pros: 
> (Bruno) Tools like Boxgrinder and Oz can deal with multiple OSes and can create local images as well as push them to the cloud. The build would be repeatable and would not require any effort from the end user (apart maybe providing Oracle JDK, but that would have to be the case whatever the solution). Future contributors would just need to boot their VM to get started and hopefully ease contribution.
> (Roman) I'd love if I could say
> {code}
> $ vagrant add box/up
> {code}
> and get the environment up and running without much fuzz.
> #3 Cons:
> (Cos) I think boxed environments (like VMs) are an overkill... old proven toolchain type of environment that can automatically bootstrap upon a fresh install (or update itself: think Maven model) and pull whatever apps are required in whatever form they might exist... Such approach would be more fluid than a somewhat rigid VMs, that would have to be updated periodically, versioned, etc. Another benefit of a toolchain is that BigTop packages might have to redistribute/wrap some of the tools for later use once a package is installed on a customer's system.
> There are some tools based on SRPMs that can extract a dependency list.
> Roman suggested a simpler approach of grepping and using sed/awk: 
> {code}
> $ git grep -E 'Build(Requires|-Depends):'
> {code}
> Sean was ... in favor of the idea of extracting dependencies from the control and spec files and building a script that will install the necessary tool chain. 
> Cos mentioned ... It should work, except for build-time dependencies which aren't declared in the bigtop packages (e.g. libtool, etc.). So, I'd vote for a separated list of build-time dependencies being maintained.
> Roman separated 3 distinct use cases:
> # provisioning/maintaining a long running Jenkins slave
> # provisioning a host dev environment
> # provisioning a VM dev environment
> Roman also suggested ... a 'virtual' package called build so that running:
> {code}
> $ make build-[rpm|deb]
> {code}
> will create a package that would download all the bits and pieced of a tool-chain AND also promote BuildRequires: to its own Requires: so that installing this package will give you all the toolchain bits and all the package deps at the same time.
> The consensus seems to be that all approaches are valid for some use cases, so implementation may start from simpler approaches, and evolve as necessary.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira