Re: permanent builds on jenkins

[Konstantin Boudnik <co...@apache.org>]

And also it is 'permanent'.

On Sun, Jul 07, 2013 at 02:37PM, Konstantin Boudnik wrote:
> [Moving user@ to Bcc: and cross-posting to dev@ This is a more dev discussion,
> let's keep it there]
> 
> I think there's a confusion of some sort here. Apache normally doesn't provide
> binary releases. A release - in ASF - is a source code. While binaries can be
> provided as a convenience artifacts it is most often isn't a case.
> 
> Now, a Bigtop release is a fixed BOM file that states what versions of what
> components were built and tested to work against each other. If you are
> looking for particular system packages - it is very easy to get by 
>   make rpm|deb
> from a particular release tag. I believe we do publish binary packages for
> each release, but you should understand that these are stored on s3 that costs
> money. Right now, BT build infrastructure is running on the funds donated by
> commercial vendors (Cloudera is the only one so far, others are chiming in as
> we speak).
> 
> Same goes for a VM: if you need a VM with particular release bits in it: you
> just check out the tag and run boxgrinder that will create identical VM each
> time you run the build. The VM in question isn't meant for development of BT.
> This is a mere test-bench for people who want to try the latest release of
> Hadoop based stack and have no skills of going through the whole process by
> themselves. 
> 
> I wish we could keep a copy of VMs for all stable builds, but this is a simple
> matter of the costs vs. convenience. 
> 
> As for #3 below: this is pretty much how our Jenkins infra works. There's a
> bunch of slave VMs that are used for package building/installation/testing. If
> you have some improvements in mind - feel free to open a JIRA tickets and
> contribute a patch/design document/etc. Any help is highly welcome.
>  
> Cos
> 
> On Sun, Jul 07, 2013 at 12:03PM, Jay Vyas wrote:
> > Hi bruno:
> > 
> > 1) I've noticed that https://builds.apache.org/view/A-F/view/Bigtop/ is not
> > valid but "https://builds.apache.org/job/Bigtop-trunk/" seems to work.  Is
> > this an error on the website?
> > 
> > 2) http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/ Seems to build
> > direct from HEAD.
> > 
> > Are there actual bigtop releases which aren't on jenkins?  Everything seems
> > to be centered around the build server, rather than actual frozen,
> > downloadable releases.
> > 
> > 3) Regarding testing the VMs.  Maybe to start some simple virt-install
> > scripts could help confirm that the KVM builds, at least, are working.  If
> > we could get them working with static IPs then we could even clone down the
> > bigtop source code and run bigtop hadoop tests on the VMs after
> > construction as a validation step.
> > 
> > 
> > 
> > 
> > On Sat, Jul 6, 2013 at 8:58 PM, Bruno MahИ <bm...@apache.org> wrote:
> > 
> > > On 07/06/2013 07:33 AM, Jay Vyas wrote:
> > >
> > >> Hi bigtop:
> > >>
> > >> Are there any permanant builds saved on jenkins (for the VM matrix)?
> > >>
> > >> If not it would be nice to add them for certain known well tested,
> > >> working disk images .
> > >>
> > >> (for context, I'm currently running Mr2 build of the KVM box and it
> > >> appears to have some intermittent write issues on the DataNode path, and
> > >> also, my namenode appears to really like being in safe mode.  these
> > >> could just be due to VM setup though, as im changing some things like
> > >> adding static IPs and data node write paths... so nothing to be alarmed
> > >> about.)
> > >>
> > >> --
> > >> Jay Vyas
> > >> http://jayunit100.blogspot.com
> > >>
> > >
> > > Hi Jay,
> > >
> > > Could you defined "permanent build" ?
> > > I am not sure if this fits your requirement, but jenkins has a link to the
> > > latest successful build (ex: http://bigtop01.cloudera.org:**
> > > 8080/job/Bigtop-VM-matrix/BR=**master,KIND=kvm,label=**
> > > fedora16/lastSuccessfulBuild/**artifact/bigtop-vm-kvm-master.**tar.gz<http://bigtop01.cloudera.org:8080/job/Bigtop-VM-matrix/BR=master,KIND=kvm,label=fedora16/lastSuccessfulBuild/artifact/bigtop-vm-kvm-master.tar.gz>)
> > >
> > > We do not store convenient artifacts of VMs since no one has asked about
> > > it before.
> > > So ideally, known well tested working disk images would be the ones from
> > > Apache Bigtop releases. But right now, there is not much testing of our
> > > VMs. But any help on that front would be welcome!
> > >
> > > Note also that I added that VM more as a base VM for an Apache Hadoop
> > > cloud image than a developer VM. That's why there is not much in it as well
> > > as no desktop pre-configured.
> > > So depending on your needs, we may want to add a new VM or enhance the
> > > current one (also, boxgrinder enables inheritance between appliances).
> > >
> > > Also Boxgrinder is apparently not being maintained anymore. So we may want
> > > to look into other VM builders (Oz, etc.)
> > >
> > >
> > > Thanks,
> > > Bruno
> > >
> > 
> > 
> > 
> > -- 
> > Jay Vyas
> > http://jayunit100.blogspot.com