You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by kf...@collab.net on 2005/11/28 21:09:48 UTC
Helping CollabNet provide an automated build farm for Subversion.
Just to let you all know: I'm working with CollabNet's operations team
to set up a small build/test farm for Subversion. Erik Hülsmann first
proposed this idea privately at EuroOSCON in Amsterdam, and when we
took it to CollabNet they were in favor right away.
The goal is to have stable, consistent environments for verifying that
releases (and bindings) work on certain platforms. Right now we've
got a box each allocated for Linux (probably RHEL4 or FC4) and Windows
XP; next in line are Solaris 10 and Mac OS X, but we'd like to get the
Linux and Windows instances up and running first.
WHAT YOU CAN DO TO HELP:
These boxes will be managed by CollabNet's operations team, but
they'll be configured according a specification that the Subversion
project writes. I'll be starting that specification in our www/ area;
anyone who has experience with such systems, please help out. The
better our spec, the more easily (== the more quickly) CollabNet can
set up the farm.
Erik Hülsmann, Sander Striker and I discussed various build/test
systems in Amsterdam, and came up with this short list of candidates
(Erik, Sander, did I leave out anything major?):
* BuildBot
https://sourceforge.net/projects/buildbot/
http://buildbot.sourceforge.net/manual-0.7.0.html
* GUMP
http://gump.apache.org/
* Tinderbox
http://www.mozilla.org/tinderbox.html
If you have experience with any of these, or if you favor another
system, please follow up.
ABOUT TESTING UNCOMITTED CHANGES:
These boxes will not only be for testing trunk and release tags, but
for testing uncommitted changes -- say, changes that fix the bindings
on Windows when they're broken and those who have the expertise to fix
them don't have access to a Windows box (to pick a totally random
example, ahem).
Even though we won't have login access to these boxes, at least one
system, BuildBot, has a feature that allows a client instance to do a
patched build, see
http://buildbot.sourceforge.net/manual-0.7.0.html#try
(Thanks to Erik for digging that up.)
Because of this feature, and because of BuildBot's documentation and
portability (see http://buildbot.videolan.org/), I'm leaning toward
BuildBot right now. However, real-world experience is the best data,
so if anyone here has any, this is the time to speak up.
Thanks,
-Karl
--
www.collab.net <> CollabNet | Distributed Development On Demand
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by Daniel Rall <dl...@collab.net>.
In addition to that "testing uncommitted changes" offered by BuildBot,
the whole master/slave model will be really nice to have (pointed out
by...Erik?).
I've had pretty good experiences with CruiseControl (with Subversion),
but I unaware of it handling either of those two killer features.
--
Daniel Rall
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by kf...@collab.net.
Branko Čibej <br...@xbc.nu> writes:
> Unless buildbot is noticeably worse in other areas, the
> uncommitted-change thingy is a huge feature. Other than that, there
> are two "must" features that such a build tool should have:
>
> * It must integrate well enough with SVN to know when to skip a
> build due to there being no changes in the repository.
I know that it integrates with SVN (the documentation talks about it);
I'll soon know if it integrates this well. I suspect it does, but if
not, we can probably fix it.
> * It must be able to send mail to everyone who committed a change
> that may have broken a build (that is, everyone who committed
> since the last successful build).
>
> The second item is important because (I suspect) many committers still
> don't subscribe to svn-breakage.
Well, that I *know* we can hack in if needed :-), but again, the docs
I skimmed today seem to imply that BuildBot can already do this.
-Karl
--
www.collab.net <> CollabNet | Distributed Development On Demand
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by Branko Čibej <br...@xbc.nu>.
kfogel@collab.net wrote:
> Just to let you all know: I'm working with CollabNet's operations team
> to set up a small build/test farm for Subversion. Erik Hülsmann first
> proposed this idea privately at EuroOSCON in Amsterdam, and when we
> took it to CollabNet they were in favor right away.
>
> The goal is to have stable, consistent environments for verifying that
> releases (and bindings) work on certain platforms. Right now we've
> got a box each allocated for Linux (probably RHEL4 or FC4) and Windows
> XP; next in line are Solaris 10 and Mac OS X, but we'd like to get the
> Linux and Windows instances up and running first.
>
> WHAT YOU CAN DO TO HELP:
>
> These boxes will be managed by CollabNet's operations team, but
> they'll be configured according a specification that the Subversion
> project writes. I'll be starting that specification in our www/ area;
> anyone who has experience with such systems, please help out. The
> better our spec, the more easily (== the more quickly) CollabNet can
> set up the farm.
>
> Erik Hülsmann, Sander Striker and I discussed various build/test
> systems in Amsterdam, and came up with this short list of candidates
> (Erik, Sander, did I leave out anything major?):
>
> * BuildBot
> https://sourceforge.net/projects/buildbot/
> http://buildbot.sourceforge.net/manual-0.7.0.html
>
> * GUMP
> http://gump.apache.org/
>
> * Tinderbox
> http://www.mozilla.org/tinderbox.html
>
I don't know about the other two, but keep away from tinderbox. I tried
installing it once, and it's an even worse mess than Bugzilla.
(Unless you like weeks of pain and sleepless nights, in which case, be
my guest. :)
> If you have experience with any of these, or if you favor another
> system, please follow up.
>
I've been using CruiseControl (well, CC.Net on Windows, really) --
http://cruisecontrol.sourceforge.net/ -- and found it quite nice. I
don't think it can handle uncommitted changes, though.
> ABOUT TESTING UNCOMITTED CHANGES:
>
> These boxes will not only be for testing trunk and release tags, but
> for testing uncommitted changes -- say, changes that fix the bindings
> on Windows when they're broken and those who have the expertise to fix
> them don't have access to a Windows box (to pick a totally random
> example, ahem).
>
> Even though we won't have login access to these boxes, at least one
> system, BuildBot, has a feature that allows a client instance to do a
> patched build, see
>
> http://buildbot.sourceforge.net/manual-0.7.0.html#try
>
> (Thanks to Erik for digging that up.)
>
> Because of this feature, and because of BuildBot's documentation and
> portability (see http://buildbot.videolan.org/), I'm leaning toward
> BuildBot right now. However, real-world experience is the best data,
> so if anyone here has any, this is the time to speak up.
>
Unless buildbot is noticeably worse in other areas, the
uncommitted-change thingy is a huge feature. Other than that, there are
two "must" features that such a build tool should have:
* It must integrate well enough with SVN to know when to skip a
build due to there being no changes in the repository.
* It must be able to send mail to everyone who committed a change
that may have broken a build (that is, everyone who committed
since the last successful build).
The second item is important because (I suspect) many committers still
don't subscribe to svn-breakage.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by kf...@collab.net.
Erik Huelsmann <eh...@gmail.com> writes:
> What I turned to Collab for is hosting a coordination server: a
> central instance of a program which coordinates which build servers
> need to run which tests - collecting the test results and making them
> available to us developers.
That's a good point, thanks Erik.
CollabNet has offered up some of the build clients too, but we can,
and should, also integrate clients donated by other organizations.
(The people at ThoughtWorks had expressed interest in doing this some
time back, we'll ping them again soon.) One of the goals of writing
this spec publicly, in fact, is to make it easy for anyone to
configure a client into the farm.
-Karl
--
www.collab.net <> CollabNet | Distributed Development On Demand
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by Erik Huelsmann <eh...@gmail.com>.
On 11/28/05, kfogel@collab.net <kf...@collab.net> wrote:
> Just to let you all know: I'm working with CollabNet's operations team
> to set up a small build/test farm for Subversion. Erik Hülsmann first
> proposed this idea privately at EuroOSCON in Amsterdam, and when we
> took it to CollabNet they were in favor right away.
I did. Though I wasn't primarily targetting Collab, I noticed people
wanting to help the project: donating back to where they got such a
valuable resource from. Most organisations don't have the resources
to donate coding time. As whe don't (yet?) have a bank account,
accepting money isn't an option. So, I asked myself: what can
people/organisations afford when they want to help out in the project.
Most organisations will be able to donate a machine to run automated
tests on, possibly (hopefully) placing it outside their DMZ, so that
we can access the machines and run our testing code without them
running risks.
What I turned to Collab for is hosting a coordination server: a
central instance of a program which coordinates which build servers
need to run which tests - collecting the test results and making them
available to us developers.
bye,
Erik.
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by David Anderson <da...@calixo.net>.
kfogel@collab.net wrote:
> Just to let you all know: I'm working with CollabNet's operations team
> to set up a small build/test farm for Subversion. Erik Hülsmann first
> proposed this idea privately at EuroOSCON in Amsterdam, and when we
> took it to CollabNet they were in favor right away.
Great idea ! (and thanks to CollabNet for backing it!)
As for the build farm systems, I haven't used any myself, but I know
that OpenOffice.Org use tinderbox for their build farm. One of my
friends here is in charge of setting up the MacOS X tinderbox for OOo
(our university happens to employ the co-lead of the OSX port).
So far, his story has been one of Pain, but I am unsure if this is
related to tinderbox itself, or to difficulties getting OOo to build
properly on OSX. I'll ask him tomorrow about his experience of
Tinderbox and get back to the list on it.
Aside from that, I like the prospect of being able to do trial builds on
the build farm. This sprouts another question, not quite related to the
build farm, but more to build automation: would it be possible to set up
<insert selected build manager> to build Subversion releases as well?
Having a machine correctly setup to roll release tarballs as well as do
validation building would help streamline the release process, and avoid
stupid environment mistakes such as the one I made with 1.3.0-rc3.
It would also help by lowering the entry barrier to being a RM. Okay,
it only took one evening to get the build environment setup, but I'd
have preferred a big red "Roll it!" button and a big green "Upload and
announce it!" button at the time.
Do any of the shortlisted systems allow this? (possibly with some
tweaking, which could be provided by Subversion developers)
- Dave.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by Molle Bestefich <mo...@gmail.com>.
Miha Vitorovic wrote:
> I think a VM server (such as vmware) would
> come real handy in a case like this.
Good point - probably a lot cheaper to buy VM server software than to
coordinate servers located all around the world with a whole bunch of
different points of failure in between them and the people doing the
coordination.
Virtual Server 2005 costs 100 US$ in 4-cpu version, plus the WinXP or
Win2003 license:
http://www.microsoft.com/windowsserversystem/virtualserver/howtobuy/default.mspx#EEAA
It's 200 $ for the unlimited-cpu version. Runs Windows and I'm quite
sure it runs Linux too. Not sure whether it runs newer FreeBSD.
You'll definitely have problems with more exotic stuff like older
BSDs.
VMware GSX Server is 1400 $ for 2-cpu version, 2800 $ for unlimited
cpu. Workstation is 189 $, free if you just need they player. Either
version will run just about anything you throw at it.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Helping CollabNet provide an automated build farm for Subversion.
Posted by Miha Vitorovic <mv...@nil.si>.
I'm not a SVN developer, but I have a suggestion anyway (maybe you've
thought of that already) -- I think a VM server (such as vmware) would
come real handy in a case like this.
- First, you only need physical one box, a bit stronger, but still just
one box.
- Second, once you have a certain clean system prepared, you just shut it
down, and copy the image to a safe location. From then on you can always
get a clean box without having to re-install everything.
- Third, it's easy to add another build system - no need to buy new
hardware.
Cheers,
---
Miha Vitorovic
Inženir v tehničnem področju
Customer Support Engineer
NIL Data Communications, Tivolska cesta 48, 1000 Ljubljana, Slovenia
Phone +386 1 4746 500 Fax +386 1 4746 501 http://www.NIL.si
kfogel@collab.net wrote on 28.11.2005 22:09:48:
> The goal is to have stable, consistent environments for verifying that
> releases (and bindings) work on certain platforms. Right now we've
> got a box each allocated for Linux (probably RHEL4 or FC4) and Windows
> XP; next in line are Solaris 10 and Mac OS X, but we'd like to get the
> Linux and Windows instances up and running first.
>
> WHAT YOU CAN DO TO HELP:
>
> These boxes will be managed by CollabNet's operations team, but
> they'll be configured according a specification that the Subversion
> project writes. I'll be starting that specification in our www/ area;
> anyone who has experience with such systems, please help out. The
> better our spec, the more easily (== the more quickly) CollabNet can
> set up the farm.