You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Jason Dillon <ja...@planet57.com> on 2006/12/02 03:53:43 UTC
Geronimo build automation status (longish)
Sorry, this has been long overdue. I've been working on some
automation systems for Geronimo builds, including the basic server
assemblies, cts assemblies, tck testsuite execution as well as soon
to run our own testsuite.
I have used many different build automation platforms in the past...
but IMO they all have some deficiency. Anyways, I elected to
implement a solution using AntHill, who's publisher, Urbancode, has a
policy to allow free usage for open-source projects (just like
Atlassian's JIRA & Confluence).
I've set up the latest version of AntHill 3 on our gbuild hosts, and
have been working on a configuration to allow reliable builds of
Geronimo. One of the nice aspects of AntHill3 is its distributed
agent system, which allows workload to be split up over a set of
nodes. A downside to this is that it becomes more difficult to link
Maven builds, as Maven uses a local repository cache for all
integration points. But, I have gotten around this issue by having
AH include all of the artifacts downloaded and produced by a build
into a clean local repo by the target project which is building.
A nice side effect of this is that there is direct correlation
between one build to another. And aside from any mystical SNAPSHOT
mismatches (which I hope to get fixed soon with my mvn patch http://
jira.codehaus.org/browse/MNG-2681) it is fairly safe to say that
artifacts generated/downloaded by one build will be used by a
dependent build. The down side to this is that sometimes we have to
ship about ~512mb of dependencies for larger builds (like the cts-
server builds for the TCK which depend on all of the outputs of the
server builds, which is a local repo cache of ~512mb).
An even nicer side effect to all of this, now that each build has a
set of artifacts which can be retrieved by another process... we can
then take a successful build of Geronimo and run our testsuite on
it... either automatically or manually. And when the testsuite gets
bigger and bigger, we can split up each of the suites and run each
one a different system... or even on a different operating system or
architecture.
Anyways... the options ahead of us are really interesting... and I
believe that right now that AntHill3 is the best tool available to
our community to build a really rich and powerful build automation
system.
I am however still working out some of the kinks...
For example, to run our console-testsuite automatically on gbuild
hosts, we need to setup a virtual X server for Firefox to connect to,
which means we need to setup some tasks to execute Xfvb before tests
and shut it down afterwards, as well as put firefox-bin on the path,
etc. Minor issues, but still work left to be done.
If you'd like to take a peek, you can log into the AntHill console here:
https://gbuild.org:9443
Username: guest
Password: gbuild
(NOTE: This user will not be able to see any of the CTS or TCK
related projects due to NDA mucky muck)
I hope to have this wrapped up for the main G server builds over the
next few days, at which point I will enable the build status
notifications to be sent to dev@ But right now since I am testing
its probably not meaningful to send out those notifications.
But, I have found several build related issues from testing this
system, which is usually performed off of a clean svn co with a clean
mvn repo... so I'm confident that once its going that we will catch
more errors faster, which will hopefully reduce build related errors
for the masses.
* * *
Anyways, right now I have builds setup for:
Genesis - trunk
Specs - trunk
Geronimo Components (stage=bootstrap) - trunk & 1.2
OpenEJB 2 - trunk & 2.2
Geronimo Server (stage=assemble) - trunk & 1.2
Geronimo CTS 1.2
As noted above, these builds use the exact outputs from one build in
another, not using a shared local repo, so there is less chance that
other builds will cause mvn to behave strangely (or stranger than it
already does).
I have started working on a workflow to run the server/testsuite/*
modules on "Geronimo Server" outputs, which should be closer to being
finished early next week.
Some of these projects, those that generate Surefire reports, will
have a "Surefire_Report" attached to the buildlife. This is a
consolidated report of all the tests for that project. For example
the last build of specs trunk, had 300 tests (all passed).
Re: Geronimo build automation status (longish)
Posted by Matt Hogstrom <ma...@hogstrom.org>.
On Dec 28, 2006, at 12:11 AM, Jason Dillon wrote:
> A little holiday update on build automation muck...
>
> Turns out jolly ole st. nick brought me some coal for all of this
> anthill effort. I had thought that it would be easier/faster to get
> automated TCK builds up and running with direct effect than it would
> be to put more time into the GBuild codebase.
I got a cool coffee mug from my son ... beats your coal hands down.
>
> Its still not all anthill's fault though... mvn really doesn't make
> life any easier with its limited release capabilities and lack of
> artifact promotion, network dependency and other problems. Really...
> making it really difficult to architect a rich and robust build
> automation solution around it... impossible even.
Edison failed at many attempts to make a light bulb before he succeeded.
> In retrospect it would have been more worthwhile to work on
> integrating gbuild into continuum and adding more workflow features.
> Oh well... now I'm just so sick of all things build that I'm not
> really sure what I'm gonna do.
I think we've left you flapping in the wind for a while now and all
alone taboot. Apart from the spec thing that set you back it sounds
like the current significant problems are:
Stable repo (yes, you've mentioned it many times)
- Sounds like there are really several issues here. First is
availability which is something that we need to address with the
Infra folks. Second is change to artifacts. The XMLBeans problem
does hilight the issue with SNAPSHOTs. Moving to a private repo does
improve buildability but also introduces the problem of staying
current and that then becomes an administrative issue.
I hacked some code together earlier for Maven 1 to look at our
dependencies and check for the latest versions. I spect that can be
enhanced for M2 and it could provide a nightly report on SNAPSHOTs
that have changed. I can take a peek at that if it sounds like that
would help.
> Ho ho ho... click.
>
> :-\
>
> --jason
>
>
> On 12/1/06, Jason Dillon <ja...@planet57.com> wrote:
>> Sorry, this has been long overdue. I've been working on some
>> automation systems for Geronimo builds, including the basic server
>> assemblies, cts assemblies, tck testsuite execution as well as soon
>> to run our own testsuite.
>>
>> I have used many different build automation platforms in the past...
>> but IMO they all have some deficiency. Anyways, I elected to
>> implement a solution using AntHill, who's publisher, Urbancode, has a
>> policy to allow free usage for open-source projects (just like
>> Atlassian's JIRA & Confluence).
>>
>> I've set up the latest version of AntHill 3 on our gbuild hosts, and
>> have been working on a configuration to allow reliable builds of
>> Geronimo. One of the nice aspects of AntHill3 is its distributed
>> agent system, which allows workload to be split up over a set of
>> nodes. A downside to this is that it becomes more difficult to link
>> Maven builds, as Maven uses a local repository cache for all
>> integration points. But, I have gotten around this issue by having
>> AH include all of the artifacts downloaded and produced by a build
>> into a clean local repo by the target project which is building.
>>
>> A nice side effect of this is that there is direct correlation
>> between one build to another. And aside from any mystical SNAPSHOT
>> mismatches (which I hope to get fixed soon with my mvn patch http://
>> jira.codehaus.org/browse/MNG-2681) it is fairly safe to say that
>> artifacts generated/downloaded by one build will be used by a
>> dependent build. The down side to this is that sometimes we have to
>> ship about ~512mb of dependencies for larger builds (like the cts-
>> server builds for the TCK which depend on all of the outputs of the
>> server builds, which is a local repo cache of ~512mb).
>>
>> An even nicer side effect to all of this, now that each build has a
>> set of artifacts which can be retrieved by another process... we can
>> then take a successful build of Geronimo and run our testsuite on
>> it... either automatically or manually. And when the testsuite gets
>> bigger and bigger, we can split up each of the suites and run each
>> one a different system... or even on a different operating system or
>> architecture.
>>
>> Anyways... the options ahead of us are really interesting... and I
>> believe that right now that AntHill3 is the best tool available to
>> our community to build a really rich and powerful build automation
>> system.
>>
>> I am however still working out some of the kinks...
>>
>> For example, to run our console-testsuite automatically on gbuild
>> hosts, we need to setup a virtual X server for Firefox to connect to,
>> which means we need to setup some tasks to execute Xfvb before tests
>> and shut it down afterwards, as well as put firefox-bin on the path,
>> etc. Minor issues, but still work left to be done.
>>
>> If you'd like to take a peek, you can log into the AntHill console
>> here:
>>
>> https://gbuild.org:9443
>>
>> Username: guest
>> Password: gbuild
>>
>> (NOTE: This user will not be able to see any of the CTS or TCK
>> related projects due to NDA mucky muck)
>>
>> I hope to have this wrapped up for the main G server builds over the
>> next few days, at which point I will enable the build status
>> notifications to be sent to dev@ But right now since I am testing
>> its probably not meaningful to send out those notifications.
>>
>> But, I have found several build related issues from testing this
>> system, which is usually performed off of a clean svn co with a clean
>> mvn repo... so I'm confident that once its going that we will catch
>> more errors faster, which will hopefully reduce build related errors
>> for the masses.
>>
>> * * *
>>
>> Anyways, right now I have builds setup for:
>>
>> Genesis - trunk
>> Specs - trunk
>> Geronimo Components (stage=bootstrap) - trunk & 1.2
>> OpenEJB 2 - trunk & 2.2
>> Geronimo Server (stage=assemble) - trunk & 1.2
>> Geronimo CTS 1.2
>>
>> As noted above, these builds use the exact outputs from one build in
>> another, not using a shared local repo, so there is less chance that
>> other builds will cause mvn to behave strangely (or stranger than it
>> already does).
>>
>> I have started working on a workflow to run the server/testsuite/*
>> modules on "Geronimo Server" outputs, which should be closer to being
>> finished early next week.
>>
>> Some of these projects, those that generate Surefire reports, will
>> have a "Surefire_Report" attached to the buildlife. This is a
>> consolidated report of all the tests for that project. For example
>> the last build of specs trunk, had 300 tests (all passed).
>>
>>
>>
>>
>> NOTE: Screen shots attached, as currently the state of the install
>> may change as I configure to validate the setup.
>>
>> You can also see & download any of the outputs of the build.
>>
>>
>>
>>
>> * * *
>>
>> Anyways, as mentioned this all needs a little bit more love to be
>> more of the perfect build automation system which was what I have
>> been really trying to put together for our community.
>>
>> Should have at the very least, nightly deploys of SNAPSHOTs hooked up
>> for G projects by early next week. Then nightly website updates, and
>> then automated testsuite & tck bits will follow shortly afterwards...
>> and eventually, we could also use AH to generate the RC and release
>> builds, so that all builds are generated from the same environment.
>> But probably sooner than that, we can promote G server builds that
>> pass the TCK or our Testsuite, so that the exact binaries used to
>> build the CTS server or run the testsuite can be used by others for
>> more validation.
>>
>> I will send out email again later, with some brief wiki docs on what
>> all this AH jargon is, and how to spin off builds with a few clicks.
>>
>> --jason
>>
>>
>>
>>
>>
>>
>>
>
Matt Hogstrom
matt@hogstrom.org
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
A little holiday update on build automation muck...
Turns out jolly ole st. nick brought me some coal for all of this
anthill effort. I had thought that it would be easier/faster to get
automated TCK builds up and running with direct effect than it would
be to put more time into the GBuild codebase.
Well, months have gone by now and I'm still no where near to getting
anthill to behave... and I'm close to giving up on it. Still gonna
see if the anthill folks can get some of the issues I've been
reporting to them fixed, and they have been very good about fixing
them and even adding a few features to support or TCK testsuite
automation. But if not... well I can't handle the frustration
anymore... as this has already turned me into a ragged old man whining
and bitching at everyone.
Its still not all anthill's fault though... mvn really doesn't make
life any easier with its limited release capabilities and lack of
artifact promotion, network dependency and other problems. Really...
making it really difficult to architect a rich and robust build
automation solution around it... impossible even.
In retrospect it would have been more worthwhile to work on
integrating gbuild into continuum and adding more workflow features.
Oh well... now I'm just so sick of all things build that I'm not
really sure what I'm gonna do.
Ho ho ho... click.
:-\
--jason
On 12/1/06, Jason Dillon <ja...@planet57.com> wrote:
> Sorry, this has been long overdue. I've been working on some
> automation systems for Geronimo builds, including the basic server
> assemblies, cts assemblies, tck testsuite execution as well as soon
> to run our own testsuite.
>
> I have used many different build automation platforms in the past...
> but IMO they all have some deficiency. Anyways, I elected to
> implement a solution using AntHill, who's publisher, Urbancode, has a
> policy to allow free usage for open-source projects (just like
> Atlassian's JIRA & Confluence).
>
> I've set up the latest version of AntHill 3 on our gbuild hosts, and
> have been working on a configuration to allow reliable builds of
> Geronimo. One of the nice aspects of AntHill3 is its distributed
> agent system, which allows workload to be split up over a set of
> nodes. A downside to this is that it becomes more difficult to link
> Maven builds, as Maven uses a local repository cache for all
> integration points. But, I have gotten around this issue by having
> AH include all of the artifacts downloaded and produced by a build
> into a clean local repo by the target project which is building.
>
> A nice side effect of this is that there is direct correlation
> between one build to another. And aside from any mystical SNAPSHOT
> mismatches (which I hope to get fixed soon with my mvn patch http://
> jira.codehaus.org/browse/MNG-2681) it is fairly safe to say that
> artifacts generated/downloaded by one build will be used by a
> dependent build. The down side to this is that sometimes we have to
> ship about ~512mb of dependencies for larger builds (like the cts-
> server builds for the TCK which depend on all of the outputs of the
> server builds, which is a local repo cache of ~512mb).
>
> An even nicer side effect to all of this, now that each build has a
> set of artifacts which can be retrieved by another process... we can
> then take a successful build of Geronimo and run our testsuite on
> it... either automatically or manually. And when the testsuite gets
> bigger and bigger, we can split up each of the suites and run each
> one a different system... or even on a different operating system or
> architecture.
>
> Anyways... the options ahead of us are really interesting... and I
> believe that right now that AntHill3 is the best tool available to
> our community to build a really rich and powerful build automation
> system.
>
> I am however still working out some of the kinks...
>
> For example, to run our console-testsuite automatically on gbuild
> hosts, we need to setup a virtual X server for Firefox to connect to,
> which means we need to setup some tasks to execute Xfvb before tests
> and shut it down afterwards, as well as put firefox-bin on the path,
> etc. Minor issues, but still work left to be done.
>
> If you'd like to take a peek, you can log into the AntHill console here:
>
> https://gbuild.org:9443
>
> Username: guest
> Password: gbuild
>
> (NOTE: This user will not be able to see any of the CTS or TCK
> related projects due to NDA mucky muck)
>
> I hope to have this wrapped up for the main G server builds over the
> next few days, at which point I will enable the build status
> notifications to be sent to dev@ But right now since I am testing
> its probably not meaningful to send out those notifications.
>
> But, I have found several build related issues from testing this
> system, which is usually performed off of a clean svn co with a clean
> mvn repo... so I'm confident that once its going that we will catch
> more errors faster, which will hopefully reduce build related errors
> for the masses.
>
> * * *
>
> Anyways, right now I have builds setup for:
>
> Genesis - trunk
> Specs - trunk
> Geronimo Components (stage=bootstrap) - trunk & 1.2
> OpenEJB 2 - trunk & 2.2
> Geronimo Server (stage=assemble) - trunk & 1.2
> Geronimo CTS 1.2
>
> As noted above, these builds use the exact outputs from one build in
> another, not using a shared local repo, so there is less chance that
> other builds will cause mvn to behave strangely (or stranger than it
> already does).
>
> I have started working on a workflow to run the server/testsuite/*
> modules on "Geronimo Server" outputs, which should be closer to being
> finished early next week.
>
> Some of these projects, those that generate Surefire reports, will
> have a "Surefire_Report" attached to the buildlife. This is a
> consolidated report of all the tests for that project. For example
> the last build of specs trunk, had 300 tests (all passed).
>
>
>
>
> NOTE: Screen shots attached, as currently the state of the install
> may change as I configure to validate the setup.
>
> You can also see & download any of the outputs of the build.
>
>
>
>
> * * *
>
> Anyways, as mentioned this all needs a little bit more love to be
> more of the perfect build automation system which was what I have
> been really trying to put together for our community.
>
> Should have at the very least, nightly deploys of SNAPSHOTs hooked up
> for G projects by early next week. Then nightly website updates, and
> then automated testsuite & tck bits will follow shortly afterwards...
> and eventually, we could also use AH to generate the RC and release
> builds, so that all builds are generated from the same environment.
> But probably sooner than that, we can promote G server builds that
> pass the TCK or our Testsuite, so that the exact binaries used to
> build the CTS server or run the testsuite can be used by others for
> more validation.
>
> I will send out email again later, with some brief wiki docs on what
> all this AH jargon is, and how to spin off builds with a few clicks.
>
> --jason
>
>
>
>
>
>
>
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 4, 2006, at 6:53 AM, Prasad Kashyap wrote:
> If I'm getting the picture right, AH is just our solution to
> building/testing G in an automated environment. Others are free to
> continue to build G the same way they had always been building before.
Yup... pretty much.
--jason
Re: Geronimo build automation status (longish)
Posted by Prasad Kashyap <go...@gmail.com>.
If I'm getting the picture right, AH is just our solution to
building/testing G in an automated environment. Others are free to
continue to build G the same way they had always been building before.
Cheers
Prasad
On 12/4/06, John Sisson <jr...@gmail.com> wrote:
> Hi Jason,
>
> I had a quick look at the AntHill console and it looked pretty cool. My
> initial thought was whether we would be discouraging potential ISVs to
> use Geronimo as a basis of their solutions by requiring them to license
> AntHill if they want to do their own automated builds/testing of
> Geronimo (e.g. so they can build and ship their own fix releases outside
> the Apache process). The AntHill site does not list prices, so I can't
> comment on what licensing of AntHill for a non-open source version of
> Geronimo would cost.
>
> If we are always going to be able to build Geronimo and test it manually
> (without AntHill), then maybe it isn't such a biggie. Thought I'd raise
> it for discussion anyway.
>
> Regards,
> John
>
> Jason Dillon wrote:
> > Sorry, this has been long overdue. I've been working on some
> > automation systems for Geronimo builds, including the basic server
> > assemblies, cts assemblies, tck testsuite execution as well as soon to
> > run our own testsuite.
> >
> > I have used many different build automation platforms in the past...
> > but IMO they all have some deficiency. Anyways, I elected to
> > implement a solution using AntHill, who's publisher, Urbancode, has a
> > policy to allow free usage for open-source projects (just like
> > Atlassian's JIRA & Confluence).
> >
> > I've set up the latest version of AntHill 3 on our gbuild hosts, and
> > have been working on a configuration to allow reliable builds of
> > Geronimo. One of the nice aspects of AntHill3 is its distributed
> > agent system, which allows workload to be split up over a set of
> > nodes. A downside to this is that it becomes more difficult to link
> > Maven builds, as Maven uses a local repository cache for all
> > integration points. But, I have gotten around this issue by having AH
> > include all of the artifacts downloaded and produced by a build into a
> > clean local repo by the target project which is building.
> >
> > A nice side effect of this is that there is direct correlation between
> > one build to another. And aside from any mystical SNAPSHOT mismatches
> > (which I hope to get fixed soon with my mvn patch
> > http://jira.codehaus.org/browse/MNG-2681) it is fairly safe to say
> > that artifacts generated/downloaded by one build will be used by a
> > dependent build. The down side to this is that sometimes we have to
> > ship about ~512mb of dependencies for larger builds (like the
> > cts-server builds for the TCK which depend on all of the outputs of
> > the server builds, which is a local repo cache of ~512mb).
> >
> > An even nicer side effect to all of this, now that each build has a
> > set of artifacts which can be retrieved by another process... we can
> > then take a successful build of Geronimo and run our testsuite on
> > it... either automatically or manually. And when the testsuite gets
> > bigger and bigger, we can split up each of the suites and run each one
> > a different system... or even on a different operating system or
> > architecture.
> >
> > Anyways... the options ahead of us are really interesting... and I
> > believe that right now that AntHill3 is the best tool available to our
> > community to build a really rich and powerful build automation system.
> >
> > I am however still working out some of the kinks...
> >
> > For example, to run our console-testsuite automatically on gbuild
> > hosts, we need to setup a virtual X server for Firefox to connect to,
> > which means we need to setup some tasks to execute Xfvb before tests
> > and shut it down afterwards, as well as put firefox-bin on the path,
> > etc. Minor issues, but still work left to be done.
> >
> > If you'd like to take a peek, you can log into the AntHill console here:
> >
> > https://gbuild.org:9443
> >
> > Username: guest
> > Password: gbuild
> >
> > (NOTE: This user will not be able to see any of the CTS or TCK related
> > projects due to NDA mucky muck)
> >
> > I hope to have this wrapped up for the main G server builds over the
> > next few days, at which point I will enable the build status
> > notifications to be sent to dev@ But right now since I am testing its
> > probably not meaningful to send out those notifications.
> >
> > But, I have found several build related issues from testing this
> > system, which is usually performed off of a clean svn co with a clean
> > mvn repo... so I'm confident that once its going that we will catch
> > more errors faster, which will hopefully reduce build related errors
> > for the masses.
> >
> > * * *
> >
> > Anyways, right now I have builds setup for:
> >
> > Genesis - trunk
> > Specs - trunk
> > Geronimo Components (stage=bootstrap) - trunk & 1.2
> > OpenEJB 2 - trunk & 2.2
> > Geronimo Server (stage=assemble) - trunk & 1.2
> > Geronimo CTS 1.2
> >
> > As noted above, these builds use the exact outputs from one build in
> > another, not using a shared local repo, so there is less chance that
> > other builds will cause mvn to behave strangely (or stranger than it
> > already does).
> >
> > I have started working on a workflow to run the server/testsuite/*
> > modules on "Geronimo Server" outputs, which should be closer to being
> > finished early next week.
> >
> > Some of these projects, those that generate Surefire reports, will
> > have a "Surefire_Report" attached to the buildlife. This is a
> > consolidated report of all the tests for that project. For example
> > the last build of specs trunk, had 300 tests (all passed).
> >
> >
> > NOTE: Screen shots attached, as currently the state of the install may
> > change as I configure to validate the setup.
> >
> > You can also see & download any of the outputs of the build.
> >
> >
> > * * *
> >
> > Anyways, as mentioned this all needs a little bit more love to be more
> > of the perfect build automation system which was what I have been
> > really trying to put together for our community.
> >
> > Should have at the very least, nightly deploys of SNAPSHOTs hooked up
> > for G projects by early next week. Then nightly website updates, and
> > then automated testsuite & tck bits will follow shortly afterwards...
> > and eventually, we could also use AH to generate the RC and release
> > builds, so that all builds are generated from the same environment.
> > But probably sooner than that, we can promote G server builds that
> > pass the TCK or our Testsuite, so that the exact binaries used to
> > build the CTS server or run the testsuite can be used by others for
> > more validation.
> >
> > I will send out email again later, with some brief wiki docs on what
> > all this AH jargon is, and how to spin off builds with a few clicks.
> >
> > --jason
> >
> >
> >
>
>
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 4, 2006, at 8:49 PM, John Sisson wrote:
>> What? How did you get the idea that everyone has to use AntHill
>> to build Geronimo?
> I didn't have the idea that everyone has to use AntHill to build
> Geronimo. In my first paragraph I was only talking about automated
> building/testing.
> I just put my ASF hat on and asked myself who is our community?
> AFAIK our community includes Geronimo users (both individuals and
> companies), Geronimo developers and software vendors that
> repackage the code and/or provide support ). I don't currently work
> for a company that builds or sells support for Geronimo, so my only
> motivation here is to ensure there is community discussion about
> this proposed move.
You (or whoever) should, at any point, be able to run the build steps
by hand to produce the same end result... always assuming the correct
JDK, probably a non-windows environment, and a clean mvn repo with
decent network connectivity. Windows builds obviously take a bit
more care, but should also be able to run in the same manner with
some specific restrictions.
> I think my last paragraph was confusing. When I wrote it, I was
> wondering whether some time in the future, "proper" releases would
> only be built, tested and packaged from automated builds and
> whether it is realistic for someone to do a full build, test,
> package manually.
Nope, manual builds should always work assuming you have your
environment setup up correctly. The automation system simply ensure
that environment.
> Has Geronimo's building and testing and release packaging has
> become complex enough that for an ISV to realistically provide
> support for Geronimo they would pretty much need a build and
> testing automation setup like you are suggesting?
Yes, IMO the build for Geronimo, to ensure that everything is in
order, built from know components has gotten quite complex and any
ISV building in any automated fashion would want a similar setup to
ensure that everything is built using the proper dependencies. But
really any moderate to complex project would want that. And from
what I can tell none of the open solutions allow you to do that with
any degree of easy or flexibility.
> If we tell them they have to license AntHill is that reasonable?
> What does the community think?
I don't think we would ever tell anyone they *have* to use anything.
Its like, we use JIRA to manage our bugs... and then telling vendors
that if they use G, they have to use JIRA to manage their bugs...
which would never happen.
> For example, if a developer without commit access wanted to do
> automated builds and testing of modifications to Geronimo on their
> home PC or at their company (then is the AntHill license flexible
> enough to let them do that, or is the license limited to Apache
> hardware or individual Apache committers?
Such a developer could use AntHill if they wanted... or CC, or
QuickBuild, or ParaBuild, etc...
Why does it matter?
Basically, G does not deliver any automated build functionality to
its users... just as it does not deliver any issue tracking or web
content authoring tools, all of which are critical to the success of
the project.
It seems like you are lumping them up with our deliverables. I think
they are completely unrelated.
--jason
Re: Geronimo build automation status (longish)
Posted by John Sisson <jr...@gmail.com>.
Jason Dillon wrote:
> On Dec 4, 2006, at 3:45 AM, John Sisson wrote:
>> I had a quick look at the AntHill console and it looked pretty cool.
>> My initial thought was whether we would be discouraging potential
>> ISVs to use Geronimo as a basis of their solutions by requiring them
>> to license AntHill if they want to do their own automated
>> builds/testing of Geronimo (e.g. so they can build and ship their own
>> fix releases outside the Apache process). The AntHill site does not
>> list prices, so I can't comment on what licensing of AntHill for a
>> non-open source version of Geronimo would cost.
>
> What? How did you get the idea that everyone has to use AntHill to
> build Geronimo?
I didn't have the idea that everyone has to use AntHill to build
Geronimo. In my first paragraph I was only talking about automated
building/testing.
I just put my ASF hat on and asked myself who is our community? AFAIK
our community includes Geronimo users (both individuals and companies),
Geronimo developers and software vendors that repackage the code and/or
provide support ). I don't currently work for a company that builds or
sells support for Geronimo, so my only motivation here is to ensure
there is community discussion about this proposed move.
I think my last paragraph was confusing. When I wrote it, I was
wondering whether some time in the future, "proper" releases would only
be built, tested and packaged from automated builds and whether it is
realistic for someone to do a full build, test, package manually. Has
Geronimo's building and testing and release packaging has become complex
enough that for an ISV to realistically provide support for Geronimo
they would pretty much need a build and testing automation setup like
you are suggesting? If we tell them they have to license AntHill is
that reasonable? What does the community think?
For example, if a developer without commit access wanted to do automated
builds and testing of modifications to Geronimo on their home PC or at
their company (then is the AntHill license flexible enough to let them
do that, or is the license limited to Apache hardware or individual
Apache committers?
Regards,
John
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
At this point I think the AH usage is unofficial... something I have
been trying to get working. Once I get it to run our testsuite,
build all our projects and dependent components and run the tck
tests... once I get there, then I will officially recommend to the
PMC that we endorse and use AH as the official automated build too
for Geronimo.
But right now I'm still working out the details, testing how well it
will work.
I can mail the license text to the PMC in the mean-time for review if
that is what you would like.
--jason
On Dec 4, 2006, at 9:04 PM, Kevan Miller wrote:
>
> On Dec 4, 2006, at 10:03 PM, Jason Dillon wrote:
>
>>
>> As I mentioned in the initial email, the AntHill license is very
>> similar to that of JIRA or Confluence... they grant licenses for
>> use by open-source projects.
>>
>> I had hopped to get a more perfect and complete system before I
>> brought this to the community... though I'm not sure that would
>> have mattered much. Probably more that the tool is not open-
>> source is why you even mention this. If I did something similar
>> with CC or Continuum no one would even mention it. Too bad
>> neither of those tools is comprehensive enough to do the job
>> though... but maybe if we wait for a few more years they will be
>> closer.
>
> No, I wasn't really concerned about the tool not being open-source.
> More concerned about what the license is... I saw "Licensed TO
> Apache Geronimo TO BUILD OPEN-SOURCE PROJECTS ONLY". I'd like to be
> sure the license terms (restrictions, etc) are communicated with
> the group. So, we can understand the terms and ensure we follow
> them -- you never know when the big one will hit the Bay Area... :-P
>
> --kevan
Re: Geronimo build automation status (longish)
Posted by Kevan Miller <ke...@gmail.com>.
On Dec 4, 2006, at 10:03 PM, Jason Dillon wrote:
>
> As I mentioned in the initial email, the AntHill license is very
> similar to that of JIRA or Confluence... they grant licenses for
> use by open-source projects.
>
> I had hopped to get a more perfect and complete system before I
> brought this to the community... though I'm not sure that would
> have mattered much. Probably more that the tool is not open-source
> is why you even mention this. If I did something similar with CC
> or Continuum no one would even mention it. Too bad neither of
> those tools is comprehensive enough to do the job though... but
> maybe if we wait for a few more years they will be closer.
No, I wasn't really concerned about the tool not being open-source.
More concerned about what the license is... I saw "Licensed TO Apache
Geronimo TO BUILD OPEN-SOURCE PROJECTS ONLY". I'd like to be sure the
license terms (restrictions, etc) are communicated with the group.
So, we can understand the terms and ensure we follow them -- you
never know when the big one will hit the Bay Area... :-P
--kevan
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 5, 2006, at 1:35 PM, Kevan Miller wrote:
>> Sure. Did the PMC do any vetting for JIRA or Confluence?
>
> To my knowledge, those are both being run by (and licensed to) the
> ASF, not Geronimo. Am I wrong?
I believe so... though I need to check what the exact text of the
granted license is to be sure.
Though... I'm not sure that it really matters. I can get them to
reissue the license using "Apache Software Foundation" as the
licensee, looks like that is what Confluence has. Though this setup
is a little different that JIRA and Confluence, as this is only for
Geroniomo related projects... and I have not plans or intent to open
up this particular install for ASF-wide usage... though AH can
definitely support that... but I'm not heading in that direction
now... or anytime soon.
I also just asked Atlassian for the full-text of there open-source
EULA and they told me they don't have one specific for there open-
source licenses. They just pointed me at:
http://www.atlassian.com/about/licensing/license.jsp
But that page has like 14 EULAs on it.
From talking to the UrbanCode folks, they too do not have a specific
license for open-source use, but said "Apache uses JIRA, so I'm sure
a similar license would work for us too".
> As you note, AntHill is going to be very similar to both of those.
> I doubt that there's going to be an issue. And don't mean to be
> picking on you (happy to pick on somebody else :-P, I just think
> it's a good idea for this type of information to be shared with and
> evaluated by the project...
Its okay... I agree with you.
--jason
Re: Geronimo build automation status (longish)
Posted by Kevan Miller <ke...@gmail.com>.
On Dec 5, 2006, at 3:37 PM, Jason Dillon wrote:
>
>>> As I mentioned in the initial email, the AntHill license is very
>>> similar to that of JIRA or Confluence... they grant licenses for
>>> use by open-source projects.
>> Can the full text of the open source license be sent to the PMC
>> for vetting?
>
> Sure. Did the PMC do any vetting for JIRA or Confluence?
To my knowledge, those are both being run by (and licensed to) the
ASF, not Geronimo. Am I wrong?
As you note, AntHill is going to be very similar to both of those. I
doubt that there's going to be an issue. And don't mean to be picking
on you (happy to pick on somebody else :-P, I just think it's a good
idea for this type of information to be shared with and evaluated by
the project...
--kevan
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 4, 2006, at 8:57 PM, John Sisson wrote:
> Jason Dillon wrote:
> Maybe it is starting your response with the word "What?". I could
> have interpreted it completely differently to how you intended, but
> it did not make you seem friendly and approachable.
Eh.. maybe :-P
Maybe "Huh?"... "Hrm...?" "Whaddyamean?" "Wha-what?"...
Though they all seem to end up suggesting that I am surprised. Funny
that the plain english version makes it seem like I am being a prick
or something :-P
>> As I mentioned in the initial email, the AntHill license is very
>> similar to that of JIRA or Confluence... they grant licenses for
>> use by open-source projects.
> Can the full text of the open source license be sent to the PMC for
> vetting?
Sure. Did the PMC do any vetting for JIRA or Confluence?
>> I had hopped to get a more perfect and complete system before I
>> brought this to the community... though I'm not sure that would
>> have mattered much. Probably more that the tool is not open-
>> source is why you even mention this. If I did something similar
>> with CC or Continuum no one would even mention it. Too bad
>> neither of those tools is comprehensive enough to do the job
>> though... but maybe if we wait for a few more years they will be
>> closer.
>>
>> :-P
>>
>> Maybe I need an apple mail plugin to randomly insert emotions into
>> mail so that folks don't think that I am being harsh...
> Maybe :-) Everyone doesn't have the same thickness of skin.
Maaaaybbbeeeee not. :-) I know sometimes I misjudge the tone of
email, :-P, and then reply with similar tone to that which I
interpreted ;-) :-P
IMO best not to think about tone at all when reading email... more
often than not the interpreted tone is more a manifestation of your
own mood than what the text of the message actually indicates. :-)
:-)
:-P
--jason
Re: Geronimo build automation status (longish)
Posted by John Sisson <jr...@gmail.com>.
Jason Dillon wrote:
> I'm now a little bit disturbed that you now think that based on my
> response that I am not open to questions... which I really can't see
> how you came to that based on the words I used in my response.
Maybe it is starting your response with the word "What?". I could have
interpreted it completely differently to how you intended, but it did
not make you seem friendly and approachable.
>
>> While I'm on this thread, is the AntHill license specific to the
>> Apache Geronimo project? Or a general for Open Source project use? I
>> think it would be a good idea to notify the project and/or pmc if we
>> are obtaining licenses for use by the project...
>
> As I mentioned in the initial email, the AntHill license is very
> similar to that of JIRA or Confluence... they grant licenses for use
> by open-source projects.
Can the full text of the open source license be sent to the PMC for vetting?
>
> I had hopped to get a more perfect and complete system before I
> brought this to the community... though I'm not sure that would have
> mattered much. Probably more that the tool is not open-source is why
> you even mention this. If I did something similar with CC or
> Continuum no one would even mention it. Too bad neither of those
> tools is comprehensive enough to do the job though... but maybe if we
> wait for a few more years they will be closer.
>
> :-P
>
> Maybe I need an apple mail plugin to randomly insert emotions into
> mail so that folks don't think that I am being harsh...
Maybe :-) Everyone doesn't have the same thickness of skin.
Regards,
John
>
> blah... email really sucks as an effective communication medium.
>
> :-(
>
> --jason
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
> On Dec 4, 2006, at 1:50 PM, Jason Dillon wrote:
>
>> What? How did you get the idea that everyone has to use AntHill
>> to build Geronimo?
>>
>> You, or anyone are free to use any tool you like to build Geronimo
>> in any fashion you prefer. I have just found that the other build
>> automation systems available do not have enough features to fill
>> all of the different build and automated testing needs which we have.
>
> Jason,
> Will be really great to have regular builds running again (not to
> mention new testing plans in the works). Thanks!
>
> Although I agree that John was a little off-base, I can see how he
> might have gotten there... I think you could be a little more open
> to questions and slight misconceptions as people start to
> understand AntHill and the new automated build... Good news is John
> is looking at it... ;-)
I'm always open to questions...
While I was a little shocked that such a conclusion could be drawn
based on reading the mail I sent... there was no tone to be inferred
by my response whatsoever. I responded with some questions to why
the conclusions were drawn... perhaps there was something specific in
my initial email which lead him in that direction, if so I'd like to
know what those specifics are so I can clarify them.
I'm now a little bit disturbed that you now think that based on my
response that I am not open to questions... which I really can't see
how you came to that based on the words I used in my response.
Why is it that if I don't write emails with happy smily faces all
over them that people automatically assume that I am closed off,
irritated or unwilling to listen to other opinions. I am certainly
open to any comments... though I will more than likely respond to
them... in the same non-smily style.
But, for sake of clarity...
:-)
:-P
;-)
> While I'm on this thread, is the AntHill license specific to the
> Apache Geronimo project? Or a general for Open Source project use?
> I think it would be a good idea to notify the project and/or pmc if
> we are obtaining licenses for use by the project...
As I mentioned in the initial email, the AntHill license is very
similar to that of JIRA or Confluence... they grant licenses for use
by open-source projects.
I had hopped to get a more perfect and complete system before I
brought this to the community... though I'm not sure that would have
mattered much. Probably more that the tool is not open-source is why
you even mention this. If I did something similar with CC or
Continuum no one would even mention it. Too bad neither of those
tools is comprehensive enough to do the job though... but maybe if we
wait for a few more years they will be closer.
:-P
Maybe I need an apple mail plugin to randomly insert emotions into
mail so that folks don't think that I am being harsh...
blah... email really sucks as an effective communication medium.
:-(
--jason
Re: Geronimo build automation status (longish)
Posted by Kevan Miller <ke...@gmail.com>.
On Dec 4, 2006, at 1:50 PM, Jason Dillon wrote:
> What? How did you get the idea that everyone has to use AntHill to
> build Geronimo?
>
> You, or anyone are free to use any tool you like to build Geronimo
> in any fashion you prefer. I have just found that the other build
> automation systems available do not have enough features to fill
> all of the different build and automated testing needs which we have.
Jason,
Will be really great to have regular builds running again (not to
mention new testing plans in the works). Thanks!
Although I agree that John was a little off-base, I can see how he
might have gotten there... I think you could be a little more open to
questions and slight misconceptions as people start to understand
AntHill and the new automated build... Good news is John is looking
at it... ;-)
While I'm on this thread, is the AntHill license specific to the
Apache Geronimo project? Or a general for Open Source project use? I
think it would be a good idea to notify the project and/or pmc if we
are obtaining licenses for use by the project...
--kevan
Re: Geronimo build automation status (longish)
Posted by Jason Dillon <ja...@planet57.com>.
On Dec 4, 2006, at 3:45 AM, John Sisson wrote:
> I had a quick look at the AntHill console and it looked pretty
> cool. My initial thought was whether we would be discouraging
> potential ISVs to use Geronimo as a basis of their solutions by
> requiring them to license AntHill if they want to do their own
> automated builds/testing of Geronimo (e.g. so they can build and
> ship their own fix releases outside the Apache process). The
> AntHill site does not list prices, so I can't comment on what
> licensing of AntHill for a non-open source version of Geronimo
> would cost.
What? How did you get the idea that everyone has to use AntHill to
build Geronimo?
You, or anyone are free to use any tool you like to build Geronimo in
any fashion you prefer. I have just found that the other build
automation systems available do not have enough features to fill all
of the different build and automated testing needs which we have.
> If we are always going to be able to build Geronimo and test it
> manually
Yes, of course you can always build Geronimo manually. Remember the
different steps for bootstrapping? Well, AntHill basically just
automates that, and the handles passing artifacts from build to
build... but its basically the exact same set of steps which can by
run by hand.
At no point will AH, or another other automation tool be required to
build Geronimo (or any of its components)... though we may eventually
want to use an automation tool to cute nightly/weekly/release builds
so we have a higher degree of confidence due to a consistent build
environment.
> (without AntHill), then maybe it isn't such a biggie. Thought I'd
> raise it for discussion anyway.
I'm not sure what we are discussing actually.
--jason
Re: Geronimo build automation status (longish)
Posted by John Sisson <jr...@gmail.com>.
Hi Jason,
I had a quick look at the AntHill console and it looked pretty cool. My
initial thought was whether we would be discouraging potential ISVs to
use Geronimo as a basis of their solutions by requiring them to license
AntHill if they want to do their own automated builds/testing of
Geronimo (e.g. so they can build and ship their own fix releases outside
the Apache process). The AntHill site does not list prices, so I can't
comment on what licensing of AntHill for a non-open source version of
Geronimo would cost.
If we are always going to be able to build Geronimo and test it manually
(without AntHill), then maybe it isn't such a biggie. Thought I'd raise
it for discussion anyway.
Regards,
John
Jason Dillon wrote:
> Sorry, this has been long overdue. I've been working on some
> automation systems for Geronimo builds, including the basic server
> assemblies, cts assemblies, tck testsuite execution as well as soon to
> run our own testsuite.
>
> I have used many different build automation platforms in the past...
> but IMO they all have some deficiency. Anyways, I elected to
> implement a solution using AntHill, who's publisher, Urbancode, has a
> policy to allow free usage for open-source projects (just like
> Atlassian's JIRA & Confluence).
>
> I've set up the latest version of AntHill 3 on our gbuild hosts, and
> have been working on a configuration to allow reliable builds of
> Geronimo. One of the nice aspects of AntHill3 is its distributed
> agent system, which allows workload to be split up over a set of
> nodes. A downside to this is that it becomes more difficult to link
> Maven builds, as Maven uses a local repository cache for all
> integration points. But, I have gotten around this issue by having AH
> include all of the artifacts downloaded and produced by a build into a
> clean local repo by the target project which is building.
>
> A nice side effect of this is that there is direct correlation between
> one build to another. And aside from any mystical SNAPSHOT mismatches
> (which I hope to get fixed soon with my mvn patch
> http://jira.codehaus.org/browse/MNG-2681) it is fairly safe to say
> that artifacts generated/downloaded by one build will be used by a
> dependent build. The down side to this is that sometimes we have to
> ship about ~512mb of dependencies for larger builds (like the
> cts-server builds for the TCK which depend on all of the outputs of
> the server builds, which is a local repo cache of ~512mb).
>
> An even nicer side effect to all of this, now that each build has a
> set of artifacts which can be retrieved by another process... we can
> then take a successful build of Geronimo and run our testsuite on
> it... either automatically or manually. And when the testsuite gets
> bigger and bigger, we can split up each of the suites and run each one
> a different system... or even on a different operating system or
> architecture.
>
> Anyways... the options ahead of us are really interesting... and I
> believe that right now that AntHill3 is the best tool available to our
> community to build a really rich and powerful build automation system.
>
> I am however still working out some of the kinks...
>
> For example, to run our console-testsuite automatically on gbuild
> hosts, we need to setup a virtual X server for Firefox to connect to,
> which means we need to setup some tasks to execute Xfvb before tests
> and shut it down afterwards, as well as put firefox-bin on the path,
> etc. Minor issues, but still work left to be done.
>
> If you'd like to take a peek, you can log into the AntHill console here:
>
> https://gbuild.org:9443
>
> Username: guest
> Password: gbuild
>
> (NOTE: This user will not be able to see any of the CTS or TCK related
> projects due to NDA mucky muck)
>
> I hope to have this wrapped up for the main G server builds over the
> next few days, at which point I will enable the build status
> notifications to be sent to dev@ But right now since I am testing its
> probably not meaningful to send out those notifications.
>
> But, I have found several build related issues from testing this
> system, which is usually performed off of a clean svn co with a clean
> mvn repo... so I'm confident that once its going that we will catch
> more errors faster, which will hopefully reduce build related errors
> for the masses.
>
> * * *
>
> Anyways, right now I have builds setup for:
>
> Genesis - trunk
> Specs - trunk
> Geronimo Components (stage=bootstrap) - trunk & 1.2
> OpenEJB 2 - trunk & 2.2
> Geronimo Server (stage=assemble) - trunk & 1.2
> Geronimo CTS 1.2
>
> As noted above, these builds use the exact outputs from one build in
> another, not using a shared local repo, so there is less chance that
> other builds will cause mvn to behave strangely (or stranger than it
> already does).
>
> I have started working on a workflow to run the server/testsuite/*
> modules on "Geronimo Server" outputs, which should be closer to being
> finished early next week.
>
> Some of these projects, those that generate Surefire reports, will
> have a "Surefire_Report" attached to the buildlife. This is a
> consolidated report of all the tests for that project. For example
> the last build of specs trunk, had 300 tests (all passed).
>
>
> NOTE: Screen shots attached, as currently the state of the install may
> change as I configure to validate the setup.
>
> You can also see & download any of the outputs of the build.
>
>
> * * *
>
> Anyways, as mentioned this all needs a little bit more love to be more
> of the perfect build automation system which was what I have been
> really trying to put together for our community.
>
> Should have at the very least, nightly deploys of SNAPSHOTs hooked up
> for G projects by early next week. Then nightly website updates, and
> then automated testsuite & tck bits will follow shortly afterwards...
> and eventually, we could also use AH to generate the RC and release
> builds, so that all builds are generated from the same environment.
> But probably sooner than that, we can promote G server builds that
> pass the TCK or our Testsuite, so that the exact binaries used to
> build the CTS server or run the testsuite can be used by others for
> more validation.
>
> I will send out email again later, with some brief wiki docs on what
> all this AH jargon is, and how to spin off builds with a few clicks.
>
> --jason
>
>
>