You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by David Blevins <da...@visi.com> on 2005/11/22 11:44:16 UTC
gbuild: dream to reality
On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
> When I close my eyes and think big, I see a large federation
> consisting of smaller groups of machines from individuals and
> companies sharing some common building/testing infrastructure, open
> to and co-maintained by members of the community projects, building
> all our code all the time and testing it on every variety of OS, VM
> and Database imaginable....
I'm happy to say...I so nailed this. We've got it, it works, it
rocks. It's running right now!
For those who haven't been following the gbuild commit logs over the
last two weeks, I've taken Continuum, ActiveMQ, and a whole lot of
long nights and days and created a system that can push continuum
builds onto a jms queue (activemq) where they are picked up by GBuild
Agents (slimmed and extended version of continuum) running in the
gbuild network. The results are then pushed onto a topic where they
are aggregated by another agent in the gbuild network. Putting more
work on the queue is as easy as dropping a properties file in a
directory.
At the moment, have the following machines chugging away on a mix of
tck and other work:
- stan.gbuild.org
- kyle.gbuild.org
- kenny.gbuild.org
- cartman.build.org
Going to get the Chariot Solutions boxes chef.gbuild.org,
timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
A really cool aspect of this is that we can throw more machines at it
in the middle of a run. It's just jms queue/consumer underneath so
new consumers can be added at anytime! The whole thing is also
really extendible and not specific to tck testing, so we could work
in performance testing and whip up results agents to analyze the
performance data. Sky is the limit!
Off to get some sleep finally... Will send more technical information
tomorrow. Just wanted to give everyone the good news.
Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
donating hosting, boxes, and manpower to get GBuild going. Can't
wait to get the AMD boxes in the mix too. I'm really excited about
all this and can't imagine it going any better.
-David
Re: gbuild: dream to reality
Posted by Bruce Snyder <br...@gmail.com>.
On 11/22/05, David Blevins <da...@visi.com> wrote:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
> > When I close my eyes and think big, I see a large federation
> > consisting of smaller groups of machines from individuals and
> > companies sharing some common building/testing infrastructure, open
> > to and co-maintained by members of the community projects, building
> > all our code all the time and testing it on every variety of OS, VM
> > and Database imaginable....
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over the
> last two weeks, I've taken Continuum, ActiveMQ, and a whole lot of
> long nights and days and created a system that can push continuum
> builds onto a jms queue (activemq) where they are picked up by GBuild
> Agents (slimmed and extended version of continuum) running in the
> gbuild network. The results are then pushed onto a topic where they
> are aggregated by another agent in the gbuild network. Putting more
> work on the queue is as easy as dropping a properties file in a
> directory.
>
> At the moment, have the following machines chugging away on a mix of
> tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at it
> in the middle of a run. It's just jms queue/consumer underneath so
> new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical information
> tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
Wow, this is spectacular work David!!! I'll have to buy a beer or
three in L.A. next month ;-). And a HUGE thanks to everyone who was
involved in this effort!
I'd like to get involved in this so that I can understand how to add
servers and additional builds. Also, how does the resource utilization
on these machines currently look? Do we have any type of monitoring
system set up to watch that yet? If not we should consider getting
OpenNMS or Nagios (or something similar) running to handle this task.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
The Castor Project
http://www.castor.org/
Apache Geronimo
http://geronimo.apache.org/
Re: gbuild: dream to reality
Posted by Prasad Kashyap <go...@gmail.com>.
Cool David. This is awesome. Are they being published too ?
Cheers
Prasad
On 11/22/05, Davanum Srinivas <da...@gmail.com> wrote:
>
> Great job!!!!!!!
>
> On 11/22/05, David Blevins <da...@visi.com> wrote:
> > On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
> > > When I close my eyes and think big, I see a large federation
> > > consisting of smaller groups of machines from individuals and
> > > companies sharing some common building/testing infrastructure, open
> > > to and co-maintained by members of the community projects, building
> > > all our code all the time and testing it on every variety of OS, VM
> > > and Database imaginable....
> >
> > I'm happy to say...I so nailed this. We've got it, it works, it
> > rocks. It's running right now!
> >
> > For those who haven't been following the gbuild commit logs over the
> > last two weeks, I've taken Continuum, ActiveMQ, and a whole lot of
> > long nights and days and created a system that can push continuum
> > builds onto a jms queue (activemq) where they are picked up by GBuild
> > Agents (slimmed and extended version of continuum) running in the
> > gbuild network. The results are then pushed onto a topic where they
> > are aggregated by another agent in the gbuild network. Putting more
> > work on the queue is as easy as dropping a properties file in a
> > directory.
> >
> > At the moment, have the following machines chugging away on a mix of
> > tck and other work:
> >
> > - stan.gbuild.org <http://stan.gbuild.org>
> > - kyle.gbuild.org <http://kyle.gbuild.org>
> > - kenny.gbuild.org <http://kenny.gbuild.org>
> > - cartman.build.org <http://cartman.build.org>
> >
> > Going to get the Chariot Solutions boxes chef.gbuild.org<http://chef.gbuild.org>
> ,
> > timmy.gbuild.org <http://timmy.gbuild.org>, and jimmy.gbuild.org<http://jimmy.gbuild.org>running tomorrow.
> >
> > A really cool aspect of this is that we can throw more machines at it
> > in the middle of a run. It's just jms queue/consumer underneath so
> > new consumers can be added at anytime! The whole thing is also
> > really extendible and not specific to tck testing, so we could work
> > in performance testing and whip up results agents to analyze the
> > performance data. Sky is the limit!
> >
> > Off to get some sleep finally... Will send more technical information
> > tomorrow. Just wanted to give everyone the good news.
> >
> > Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> > donating hosting, boxes, and manpower to get GBuild going. Can't
> > wait to get the AMD boxes in the mix too. I'm really excited about
> > all this and can't imagine it going any better.
> >
> > -David
> >
> >
> >
>
>
> --
> Davanum Srinivas : http://wso2.com/blogs/
>
Re: gbuild: dream to reality
Posted by Davanum Srinivas <da...@gmail.com>.
Great job!!!!!!!
On 11/22/05, David Blevins <da...@visi.com> wrote:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
> > When I close my eyes and think big, I see a large federation
> > consisting of smaller groups of machines from individuals and
> > companies sharing some common building/testing infrastructure, open
> > to and co-maintained by members of the community projects, building
> > all our code all the time and testing it on every variety of OS, VM
> > and Database imaginable....
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over the
> last two weeks, I've taken Continuum, ActiveMQ, and a whole lot of
> long nights and days and created a system that can push continuum
> builds onto a jms queue (activemq) where they are picked up by GBuild
> Agents (slimmed and extended version of continuum) running in the
> gbuild network. The results are then pushed onto a topic where they
> are aggregated by another agent in the gbuild network. Putting more
> work on the queue is as easy as dropping a properties file in a
> directory.
>
> At the moment, have the following machines chugging away on a mix of
> tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at it
> in the middle of a run. It's just jms queue/consumer underneath so
> new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical information
> tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
>
> -David
>
>
>
--
Davanum Srinivas : http://wso2.com/blogs/
Re: gbuild: dream to reality
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
Sounds very cool -- congrats on getting this all together!
Aaron
On 11/23/05, David Blevins <da...@visi.com> wrote:
> Ok, here are some of the details I promised. Cc'ing Continuum as as
> much of this code as possible is destined to them. The idea is that
> we only should need to maintain a few extensions (which I spelled as
> extention in the code, oops) for the things that are specific to us
> or they don't want.
>
> BASIC PARTS
> --------------------------------------
> Executing Agent (runs builds)
> Producing Agent (creates builds)
> Results Agent (aggregates build results)
>
> Each agent consumes or produces on a queue or topic (or both). You
> can have as many of each as you need or what. They can run in the
> same process or separate processes.
> All messages are Maps that are serialized to ObjectMessages onto
> either the BUILD.TASKS queue or the BUILD.RESULTS topic.
>
> Excecuting Agent
> -------------------
> Runs continuums various continuum componentry and "update-from-scm"
> and "execute-builder" actions. When this is done, the code has been
> updated and built. Has extendible pre and post checking of incoming
> builds and outgoing build results. Specifically there is a
> BuildAgentExtension (when spelling is fixed) that has a pre(Map
> incoming) method and a post(Map incoming, Map outgoing) method.
>
> Implemented Extensions so far:
> - FileIncludeExtension: Checks the build request for a
> configurable header format. Any headers matching that format are
> assumed to point to files that should be read in and included as text
> in the response. The file must be one under the project's work
> directory.
> - HeaderIncludeExtension: Any headers that match the configured
> pattern are copied as-is from the build request to the response message.
>
> Producing Agent
> -------------------
> Scans a directory for properties files that define builds to add to
> the build queue. This part could use some work. The implementation
> that uses properties files is fine for shoving lots of work onto a
> queue in a really easy way, but this is where we also need some more
> integration with continuum as a frontend. An implementation of A
> Producing Agent could be an alternate implementation of the Continuum
> build queue that just turns the build tasks into items that go onto
> the jms build queue. Once you do something like that, you need to
> consider how things work in Continuum in regards to it's single
> threaded nature that seems to be assumed in places of the code.
>
> Results Agent
> -------------------
> Very bare bones agent that consumes from teh BUILD.RESULTS topic and
> invokes BuildResultsExtensions (when spelling is fixed). Performs no
> other work aside from invoking the extensions. We can really go nuts
> with extensions here. No limit to the ones we can creat to log,
> track, compile, report, analyze or otherwise process the results data
> in various ways.
>
> The only BuildResultsExtension so far is one that scans for entries
> in the results map matching a regex ("^include.*" is the standard).
> It takes the value of that entry and writes it into a file. The name
> of the file can be created using entries in the map as well. So for
> example:
> {project.id}/{build.name}-{date}.properties
>
>
>
> Some things we could add or do
> --------------------------------
>
> - System status topic and agent
> - A page to display system status (the status of machines online)
> - Notification topic and agent that delegates to the notification
> manager
> - Something to browse the queue and displays the contents
> - A results extension to record how long each build took (data
> already available)
> - Something to use the average time for a build definition and the
> work in the queue and estimates the total hours of work to be done
> the divides it by the number of machines online and estimates a
> completion time.
> - Make client connection a component so it can easily be shared
> between agents
> - Security: maybe ssh-keys for all agents. use SignedObject and
> SealedObject for message integrity and host verification.
> - Sort properties files when creating build definitions.
> - Have DirectoryMonitor file updated events add definitions to the
> queue. Allows for builds to be in tasks dir and us to just exec
> 'touch foo.properties' to queue up the builds inside.
> - Add gbuild agent version to message headers
>
>
>
> That's all I have time for now. Hope this can get some conversations
> started on where to go from here.
>
> -David
>
> On Nov 22, 2005, at 2:44 AM, David Blevins wrote:
>
> > On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
> >> When I close my eyes and think big, I see a large federation
> >> consisting of smaller groups of machines from individuals and
> >> companies sharing some common building/testing infrastructure,
> >> open to and co-maintained by members of the community projects,
> >> building all our code all the time and testing it on every variety
> >> of OS, VM and Database imaginable....
> >
> > I'm happy to say...I so nailed this. We've got it, it works, it
> > rocks. It's running right now!
> >
> > For those who haven't been following the gbuild commit logs over
> > the last two weeks, I've taken Continuum, ActiveMQ, and a whole lot
> > of long nights and days and created a system that can push
> > continuum builds onto a jms queue (activemq) where they are picked
> > up by GBuild Agents (slimmed and extended version of continuum)
> > running in the gbuild network. The results are then pushed onto a
> > topic where they are aggregated by another agent in the gbuild
> > network. Putting more work on the queue is as easy as dropping a
> > properties file in a directory.
> >
> > At the moment, have the following machines chugging away on a mix
> > of tck and other work:
> >
> > - stan.gbuild.org
> > - kyle.gbuild.org
> > - kenny.gbuild.org
> > - cartman.build.org
> >
> > Going to get the Chariot Solutions boxes chef.gbuild.org,
> > timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
> >
> > A really cool aspect of this is that we can throw more machines at
> > it in the middle of a run. It's just jms queue/consumer underneath
> > so new consumers can be added at anytime! The whole thing is also
> > really extendible and not specific to tck testing, so we could work
> > in performance testing and whip up results agents to analyze the
> > performance data. Sky is the limit!
> >
> > Off to get some sleep finally... Will send more technical
> > information tomorrow. Just wanted to give everyone the good news.
> >
> > Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> > donating hosting, boxes, and manpower to get GBuild going. Can't
> > wait to get the AMD boxes in the mix too. I'm really excited about
> > all this and can't imagine it going any better.
> >
> > -David
> >
> >
>
>
Re: gbuild: dream to reality
Posted by David Blevins <da...@visi.com>.
Ok, here are some of the details I promised. Cc'ing Continuum as as
much of this code as possible is destined to them. The idea is that
we only should need to maintain a few extensions (which I spelled as
extention in the code, oops) for the things that are specific to us
or they don't want.
BASIC PARTS
--------------------------------------
Executing Agent (runs builds)
Producing Agent (creates builds)
Results Agent (aggregates build results)
Each agent consumes or produces on a queue or topic (or both). You
can have as many of each as you need or what. They can run in the
same process or separate processes.
All messages are Maps that are serialized to ObjectMessages onto
either the BUILD.TASKS queue or the BUILD.RESULTS topic.
Excecuting Agent
-------------------
Runs continuums various continuum componentry and "update-from-scm"
and "execute-builder" actions. When this is done, the code has been
updated and built. Has extendible pre and post checking of incoming
builds and outgoing build results. Specifically there is a
BuildAgentExtension (when spelling is fixed) that has a pre(Map
incoming) method and a post(Map incoming, Map outgoing) method.
Implemented Extensions so far:
- FileIncludeExtension: Checks the build request for a
configurable header format. Any headers matching that format are
assumed to point to files that should be read in and included as text
in the response. The file must be one under the project's work
directory.
- HeaderIncludeExtension: Any headers that match the configured
pattern are copied as-is from the build request to the response message.
Producing Agent
-------------------
Scans a directory for properties files that define builds to add to
the build queue. This part could use some work. The implementation
that uses properties files is fine for shoving lots of work onto a
queue in a really easy way, but this is where we also need some more
integration with continuum as a frontend. An implementation of A
Producing Agent could be an alternate implementation of the Continuum
build queue that just turns the build tasks into items that go onto
the jms build queue. Once you do something like that, you need to
consider how things work in Continuum in regards to it's single
threaded nature that seems to be assumed in places of the code.
Results Agent
-------------------
Very bare bones agent that consumes from teh BUILD.RESULTS topic and
invokes BuildResultsExtensions (when spelling is fixed). Performs no
other work aside from invoking the extensions. We can really go nuts
with extensions here. No limit to the ones we can creat to log,
track, compile, report, analyze or otherwise process the results data
in various ways.
The only BuildResultsExtension so far is one that scans for entries
in the results map matching a regex ("^include.*" is the standard).
It takes the value of that entry and writes it into a file. The name
of the file can be created using entries in the map as well. So for
example:
{project.id}/{build.name}-{date}.properties
Some things we could add or do
--------------------------------
- System status topic and agent
- A page to display system status (the status of machines online)
- Notification topic and agent that delegates to the notification
manager
- Something to browse the queue and displays the contents
- A results extension to record how long each build took (data
already available)
- Something to use the average time for a build definition and the
work in the queue and estimates the total hours of work to be done
the divides it by the number of machines online and estimates a
completion time.
- Make client connection a component so it can easily be shared
between agents
- Security: maybe ssh-keys for all agents. use SignedObject and
SealedObject for message integrity and host verification.
- Sort properties files when creating build definitions.
- Have DirectoryMonitor file updated events add definitions to the
queue. Allows for builds to be in tasks dir and us to just exec
'touch foo.properties' to queue up the builds inside.
- Add gbuild agent version to message headers
That's all I have time for now. Hope this can get some conversations
started on where to go from here.
-David
On Nov 22, 2005, at 2:44 AM, David Blevins wrote:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
>> When I close my eyes and think big, I see a large federation
>> consisting of smaller groups of machines from individuals and
>> companies sharing some common building/testing infrastructure,
>> open to and co-maintained by members of the community projects,
>> building all our code all the time and testing it on every variety
>> of OS, VM and Database imaginable....
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over
> the last two weeks, I've taken Continuum, ActiveMQ, and a whole lot
> of long nights and days and created a system that can push
> continuum builds onto a jms queue (activemq) where they are picked
> up by GBuild Agents (slimmed and extended version of continuum)
> running in the gbuild network. The results are then pushed onto a
> topic where they are aggregated by another agent in the gbuild
> network. Putting more work on the queue is as easy as dropping a
> properties file in a directory.
>
> At the moment, have the following machines chugging away on a mix
> of tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at
> it in the middle of a run. It's just jms queue/consumer underneath
> so new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical
> information tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
>
> -David
>
>
Re: gbuild: dream to reality
Posted by David Blevins <da...@visi.com>.
Ok, here are some of the details I promised. Cc'ing Continuum as as
much of this code as possible is destined to them. The idea is that
we only should need to maintain a few extensions (which I spelled as
extention in the code, oops) for the things that are specific to us
or they don't want.
BASIC PARTS
--------------------------------------
Executing Agent (runs builds)
Producing Agent (creates builds)
Results Agent (aggregates build results)
Each agent consumes or produces on a queue or topic (or both). You
can have as many of each as you need or what. They can run in the
same process or separate processes.
All messages are Maps that are serialized to ObjectMessages onto
either the BUILD.TASKS queue or the BUILD.RESULTS topic.
Excecuting Agent
-------------------
Runs continuums various continuum componentry and "update-from-scm"
and "execute-builder" actions. When this is done, the code has been
updated and built. Has extendible pre and post checking of incoming
builds and outgoing build results. Specifically there is a
BuildAgentExtension (when spelling is fixed) that has a pre(Map
incoming) method and a post(Map incoming, Map outgoing) method.
Implemented Extensions so far:
- FileIncludeExtension: Checks the build request for a
configurable header format. Any headers matching that format are
assumed to point to files that should be read in and included as text
in the response. The file must be one under the project's work
directory.
- HeaderIncludeExtension: Any headers that match the configured
pattern are copied as-is from the build request to the response message.
Producing Agent
-------------------
Scans a directory for properties files that define builds to add to
the build queue. This part could use some work. The implementation
that uses properties files is fine for shoving lots of work onto a
queue in a really easy way, but this is where we also need some more
integration with continuum as a frontend. An implementation of A
Producing Agent could be an alternate implementation of the Continuum
build queue that just turns the build tasks into items that go onto
the jms build queue. Once you do something like that, you need to
consider how things work in Continuum in regards to it's single
threaded nature that seems to be assumed in places of the code.
Results Agent
-------------------
Very bare bones agent that consumes from teh BUILD.RESULTS topic and
invokes BuildResultsExtensions (when spelling is fixed). Performs no
other work aside from invoking the extensions. We can really go nuts
with extensions here. No limit to the ones we can creat to log,
track, compile, report, analyze or otherwise process the results data
in various ways.
The only BuildResultsExtension so far is one that scans for entries
in the results map matching a regex ("^include.*" is the standard).
It takes the value of that entry and writes it into a file. The name
of the file can be created using entries in the map as well. So for
example:
{project.id}/{build.name}-{date}.properties
Some things we could add or do
--------------------------------
- System status topic and agent
- A page to display system status (the status of machines online)
- Notification topic and agent that delegates to the notification
manager
- Something to browse the queue and displays the contents
- A results extension to record how long each build took (data
already available)
- Something to use the average time for a build definition and the
work in the queue and estimates the total hours of work to be done
the divides it by the number of machines online and estimates a
completion time.
- Make client connection a component so it can easily be shared
between agents
- Security: maybe ssh-keys for all agents. use SignedObject and
SealedObject for message integrity and host verification.
- Sort properties files when creating build definitions.
- Have DirectoryMonitor file updated events add definitions to the
queue. Allows for builds to be in tasks dir and us to just exec
'touch foo.properties' to queue up the builds inside.
- Add gbuild agent version to message headers
That's all I have time for now. Hope this can get some conversations
started on where to go from here.
-David
On Nov 22, 2005, at 2:44 AM, David Blevins wrote:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
>> When I close my eyes and think big, I see a large federation
>> consisting of smaller groups of machines from individuals and
>> companies sharing some common building/testing infrastructure,
>> open to and co-maintained by members of the community projects,
>> building all our code all the time and testing it on every variety
>> of OS, VM and Database imaginable....
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over
> the last two weeks, I've taken Continuum, ActiveMQ, and a whole lot
> of long nights and days and created a system that can push
> continuum builds onto a jms queue (activemq) where they are picked
> up by GBuild Agents (slimmed and extended version of continuum)
> running in the gbuild network. The results are then pushed onto a
> topic where they are aggregated by another agent in the gbuild
> network. Putting more work on the queue is as easy as dropping a
> properties file in a directory.
>
> At the moment, have the following machines chugging away on a mix
> of tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at
> it in the middle of a run. It's just jms queue/consumer underneath
> so new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical
> information tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
>
> -David
>
>
Re: gbuild: dream to reality
Posted by "Alan D. Cabrera" <li...@toolazydogs.com>.
David Blevins wrote, On 11/22/2005 2:44 AM:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
>
>> When I close my eyes and think big, I see a large federation
>> consisting of smaller groups of machines from individuals and
>> companies sharing some common building/testing infrastructure, open
>> to and co-maintained by members of the community projects, building
>> all our code all the time and testing it on every variety of OS, VM
>> and Database imaginable....
>
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over the
> last two weeks, I've taken Continuum, ActiveMQ, and a whole lot of
> long nights and days and created a system that can push continuum
> builds onto a jms queue (activemq) where they are picked up by GBuild
> Agents (slimmed and extended version of continuum) running in the
> gbuild network. The results are then pushed onto a topic where they
> are aggregated by another agent in the gbuild network. Putting more
> work on the queue is as easy as dropping a properties file in a
> directory.
>
> At the moment, have the following machines chugging away on a mix of
> tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at it
> in the middle of a run. It's just jms queue/consumer underneath so
> new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical information
> tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
This just rocks!
Regards,
Alan
Re: gbuild: dream to reality
Posted by Hiram Chirino <hi...@hiramchirino.com>.
Who hoo! go David go!
Great job.
Regards,
Hiram
On Nov 22, 2005, at 5:44 AM, David Blevins wrote:
> On Oct 26, 2005, at 2:01 AM, David Blevins wrote:
>
>> When I close my eyes and think big, I see a large federation
>> consisting of smaller groups of machines from individuals and
>> companies sharing some common building/testing infrastructure,
>> open to and co-maintained by members of the community projects,
>> building all our code all the time and testing it on every variety
>> of OS, VM and Database imaginable....
>>
>
> I'm happy to say...I so nailed this. We've got it, it works, it
> rocks. It's running right now!
>
> For those who haven't been following the gbuild commit logs over
> the last two weeks, I've taken Continuum, ActiveMQ, and a whole lot
> of long nights and days and created a system that can push
> continuum builds onto a jms queue (activemq) where they are picked
> up by GBuild Agents (slimmed and extended version of continuum)
> running in the gbuild network. The results are then pushed onto a
> topic where they are aggregated by another agent in the gbuild
> network. Putting more work on the queue is as easy as dropping a
> properties file in a directory.
>
> At the moment, have the following machines chugging away on a mix
> of tck and other work:
>
> - stan.gbuild.org
> - kyle.gbuild.org
> - kenny.gbuild.org
> - cartman.build.org
>
> Going to get the Chariot Solutions boxes chef.gbuild.org,
> timmy.gbuild.org, and jimmy.gbuild.org running tomorrow.
>
> A really cool aspect of this is that we can throw more machines at
> it in the middle of a run. It's just jms queue/consumer underneath
> so new consumers can be added at anytime! The whole thing is also
> really extendible and not specific to tck testing, so we could work
> in performance testing and whip up results agents to analyze the
> performance data. Sky is the limit!
>
> Off to get some sleep finally... Will send more technical
> information tomorrow. Just wanted to give everyone the good news.
>
> Thanks again to everyone (Simula, Mergere, Chariot, Dain) for
> donating hosting, boxes, and manpower to get GBuild going. Can't
> wait to get the AMD boxes in the mix too. I'm really excited about
> all this and can't imagine it going any better.
>
> -David
>
>