You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Nigel Daley <ni...@apache.org> on 2009/11/05 00:18:48 UTC
Re: Hudson machine utilization
Tim, the Hadoop labeled machines were not donated to ASF. Minerva,
Vesta, and a couple others (used now for buildbot) were donated to ASF.
I agree we should encourage folks to tie their linux builds to the
"Ubuntu" label (which already exists), so both minerva and vesta get
used.
We should also encourage projects (spam-assasin, ftpserver, struts,
vysper, xwork2) to move off of the Master hudson.zones.apache.org
Nige
On Oct 28, 2009, at 8:47 AM, Tim Ellison wrote:
> On 28/Oct/2009 15:13, Justin Mason wrote:
>> Well, we could move more load from hudson.zones to minerva first:
>>
>> http://hudson.zones.apache.org/hudson/computer/%28master%29/load-statistics
>> http://hudson.zones.apache.org/hudson/computer/minerva.apache.org%20%28Ubuntu%29/load-statistics
>>
>> (wow, those are good graphs!)
>
> Why do you say to do that first?
>
> At least there are times when Minerva is using both its executors.
> However, it looks like we could get by with half the current number of
> the Hadoop labeled machines without impacting anything.
>
> http://hudson.zones.apache.org/hudson/label/Hadoop/load-statistics?type=hour
>
>> We certainly should embark on a program of persuading projects to
>> schedule their jobs on both Linux and Solaris, though, to do that....
>
> Maybe we can just define a useful set of labels to sets of nodes and
> encourage people to tie builds to them rather than specific machines.
>
> Regards,
> Tim
>
>
>> On Wed, Oct 28, 2009 at 14:48, Tim Ellison <t....@gmail.com>
>> wrote:
>>> Just looking at the Hudson machine utilization at the moment.
>>> There are
>>> a number of jobs that are tied to particular machines in the
>>> queue, and
>>> a number of (hadoop-labeled) machines that are committed to tied
>>> jobs only.
>>>
>>> I realize that the machines are courteously donated etc, but is the
>>> capacity being used effectively [1]?
>>>
>>> In particular, would the Hadoop jobs be impacted if we
>>> reclassified an
>>> existing slave as general usage, and more jobs as scheduable
>>> anywhere?
>>>
>>> [1] e.g.
>>> http://hudson.zones.apache.org/hudson/computer/hadoop1%20%28Ubuntu%29/load-statistics?type=hour
>>>
>>> Regards,
>>> Tim
>>>
>>>
>>
>>
>>
Re: Hudson machine utilization
Posted by Tim Ellison <t....@gmail.com>.
On 16/Nov/2009 00:12, Justin Mason wrote:
> On Mon, Nov 16, 2009 at 00:01, Nigel Daley <nd...@yahoo-inc.com> wrote:
>> On Nov 16, 2009, at 1:59 AM, "Tim Ellison" <t....@gmail.com> wrote:
>>> On 14/Nov/2009 04:46, Nigel Daley wrote:
>>>>>> I agree we should encourage folks to tie their linux builds to the
>>>>>> "Ubuntu" label (which already exists), so both minerva and vesta get
>>>>>> used.
>>>>>>
>>>>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>>>>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>>>>> Why are minerva and vesta configured as "Leave this machine for tied
>>>>> jobs only"? I'd expect that setting for Master and Hadoop nodes, and
>>>>> let the others pick up any job.
>>>> That would be preferable, but for legacy reasons Vesta and Minerva are
>>>> left for tied jobs. This was because the Master was the only build node
>>>> for 1.5+ years and had lots and lots of build on it when we then added
>>>> Vesta and Minerva. For compatibility reasons, we set it up as is.
>>>>
>>>> Suggestions on how to change this now? How to migrate builds off
>>>> Master? Clearly the extremes are "rip the band-aid off -- builds start
>>>> failing that try to run on Master" & "big project to contact build
>>>> owners and push them to migrate".
>>> Just tie jobs to master that have dependencies there,
>>
>> How do we determine this for the 100+ jobs?
>
> I'm assuming we can ask -- all Hudson users are supposed to be subbed
> to infrastructure@ at least. Also we can change the main site
> banner....
Yep, like I say, I don't think things are especially broken at the
moment, I'm merely suggesting a soft approach to 'stop digging the hole'
before we are in too deep to get out of trouble.
Regards,
Tim
>>> and mark it for
>>> tied jobs only, and let other jobs target labels if they have specific
>>> OS/CPU requirements.
>>>
>>> I don't think anything is particularly 'broken' at the moment is it? I
>>> was just trying to understand the current set-up, and if we ask new jobs
>>> to set up a bit differently we can prevent over burdening master while
>>> leaving spare capacity elsewhere.
>>>
>>> Regards,
>>> Tim
>>
>
>
>
Re: Hudson machine utilization
Posted by Daniel Kulp <dk...@apache.org>.
On Mon November 16 2009 11:23:42 am Nigel Daley wrote:
> I think anything currently *unbound* gets run on the master since it's
> the only 'slave' that isn't reserved for tied jobs (last I looked).
So would it make sense to "untick" that tick box for vesta and/or minerva?
--
Daniel Kulp
dkulp@apache.org
http://www.dankulp.com/blog
Re: Hudson machine utilization
Posted by Nigel Daley <nd...@yahoo-inc.com>.
On Nov 16, 2009, at 3:52 PM, Tim Ellison wrote:
> On 16/Nov/2009 09:53, Jukka Zitting wrote:
>> On Mon, Nov 16, 2009 at 1:12 AM, Justin Mason <jm...@jmason.org> wrote:
>>> On Mon, Nov 16, 2009 at 00:01, Nigel Daley <nd...@yahoo-inc.com>
>>> wrote:
>>>> How do we determine this for the 100+ jobs?
>>> I'm assuming we can ask -- all Hudson users are supposed to be
>>> subbed
>>> to infrastructure@ at least. Also we can change the main site
>>> banner....
>>
>> Do we have an easy way to get a list of all the jobs running on (vs.
>> being explicitly bound to [1]) master? I volunteer to contact at
>> least
>> some of those projects and to help them migrate their builds.
>>
>> [1] http://hudson.zones.apache.org/hudson/computer/(master)/
>
> Not that I'm aware of, other than piecemeal by watching what is
> running
> there via [2]. Hopefully there's enough info in groups of build names
> to get a few projects at a time notified.
>
> [2] http://hudson.zones.apache.org/hudson/computer/%28master%29/builds
>
> Regards,
> Tim
I think anything currently *unbound* gets run on the master since it's
the only 'slave' that isn't reserved for tied jobs (last I looked).
Nige
Re: Hudson machine utilization
Posted by Tim Ellison <t....@gmail.com>.
On 16/Nov/2009 09:53, Jukka Zitting wrote:
> On Mon, Nov 16, 2009 at 1:12 AM, Justin Mason <jm...@jmason.org> wrote:
>> On Mon, Nov 16, 2009 at 00:01, Nigel Daley <nd...@yahoo-inc.com> wrote:
>>> How do we determine this for the 100+ jobs?
>> I'm assuming we can ask -- all Hudson users are supposed to be subbed
>> to infrastructure@ at least. Also we can change the main site
>> banner....
>
> Do we have an easy way to get a list of all the jobs running on (vs.
> being explicitly bound to [1]) master? I volunteer to contact at least
> some of those projects and to help them migrate their builds.
>
> [1] http://hudson.zones.apache.org/hudson/computer/(master)/
Not that I'm aware of, other than piecemeal by watching what is running
there via [2]. Hopefully there's enough info in groups of build names
to get a few projects at a time notified.
[2] http://hudson.zones.apache.org/hudson/computer/%28master%29/builds
Regards,
Tim
Re: Hudson machine utilization
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Nov 16, 2009 at 1:12 AM, Justin Mason <jm...@jmason.org> wrote:
> On Mon, Nov 16, 2009 at 00:01, Nigel Daley <nd...@yahoo-inc.com> wrote:
>> How do we determine this for the 100+ jobs?
>
> I'm assuming we can ask -- all Hudson users are supposed to be subbed
> to infrastructure@ at least. Also we can change the main site
> banner....
Do we have an easy way to get a list of all the jobs running on (vs.
being explicitly bound to [1]) master? I volunteer to contact at least
some of those projects and to help them migrate their builds.
[1] http://hudson.zones.apache.org/hudson/computer/(master)/
BR,
Jukka Zitting
Re: Hudson machine utilization
Posted by Justin Mason <jm...@jmason.org>.
On Mon, Nov 16, 2009 at 00:01, Nigel Daley <nd...@yahoo-inc.com> wrote:
>
> Sent from my iPhone
>
> On Nov 16, 2009, at 1:59 AM, "Tim Ellison" <t....@gmail.com> wrote:
>
>> On 14/Nov/2009 04:46, Nigel Daley wrote:
>>>>>
>>>>> I agree we should encourage folks to tie their linux builds to the
>>>>> "Ubuntu" label (which already exists), so both minerva and vesta get
>>>>> used.
>>>>>
>>>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>>>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>>>>
>>>> Why are minerva and vesta configured as "Leave this machine for tied
>>>> jobs only"? I'd expect that setting for Master and Hadoop nodes, and
>>>> let the others pick up any job.
>>>
>>> That would be preferable, but for legacy reasons Vesta and Minerva are
>>> left for tied jobs. This was because the Master was the only build node
>>> for 1.5+ years and had lots and lots of build on it when we then added
>>> Vesta and Minerva. For compatibility reasons, we set it up as is.
>>>
>>> Suggestions on how to change this now? How to migrate builds off
>>> Master? Clearly the extremes are "rip the band-aid off -- builds start
>>> failing that try to run on Master" & "big project to contact build
>>> owners and push them to migrate".
>>
>> Just tie jobs to master that have dependencies there,
>
> How do we determine this for the 100+ jobs?
I'm assuming we can ask -- all Hudson users are supposed to be subbed
to infrastructure@ at least. Also we can change the main site
banner....
--j.
>> and mark it for
>> tied jobs only, and let other jobs target labels if they have specific
>> OS/CPU requirements.
>>
>> I don't think anything is particularly 'broken' at the moment is it? I
>> was just trying to understand the current set-up, and if we ask new jobs
>> to set up a bit differently we can prevent over burdening master while
>> leaving spare capacity elsewhere.
>>
>> Regards,
>> Tim
>
>
--
--j.
Re: Hudson machine utilization
Posted by Nigel Daley <nd...@yahoo-inc.com>.
Sent from my iPhone
On Nov 16, 2009, at 1:59 AM, "Tim Ellison" <t....@gmail.com>
wrote:
> On 14/Nov/2009 04:46, Nigel Daley wrote:
>>>> I agree we should encourage folks to tie their linux builds to the
>>>> "Ubuntu" label (which already exists), so both minerva and vesta
>>>> get
>>>> used.
>>>>
>>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>>>
>>> Why are minerva and vesta configured as "Leave this machine for tied
>>> jobs only"? I'd expect that setting for Master and Hadoop nodes,
>>> and
>>> let the others pick up any job.
>>
>> That would be preferable, but for legacy reasons Vesta and Minerva
>> are
>> left for tied jobs. This was because the Master was the only build
>> node
>> for 1.5+ years and had lots and lots of build on it when we then
>> added
>> Vesta and Minerva. For compatibility reasons, we set it up as is.
>>
>> Suggestions on how to change this now? How to migrate builds off
>> Master? Clearly the extremes are "rip the band-aid off -- builds
>> start
>> failing that try to run on Master" & "big project to contact build
>> owners and push them to migrate".
>
> Just tie jobs to master that have dependencies there,
How do we determine this for the 100+ jobs?
Nigel
> and mark it for
> tied jobs only, and let other jobs target labels if they have specific
> OS/CPU requirements.
>
> I don't think anything is particularly 'broken' at the moment is
> it? I
> was just trying to understand the current set-up, and if we ask new
> jobs
> to set up a bit differently we can prevent over burdening master while
> leaving spare capacity elsewhere.
>
> Regards,
> Tim
Re: Hudson machine utilization
Posted by Justin Mason <jm...@jmason.org>.
On Sun, Nov 15, 2009 at 20:28, Tim Ellison <t....@gmail.com> wrote:
> On 14/Nov/2009 04:46, Nigel Daley wrote:
>>>> I agree we should encourage folks to tie their linux builds to the
>>>> "Ubuntu" label (which already exists), so both minerva and vesta get
>>>> used.
>>>>
>>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>>>
>>> Why are minerva and vesta configured as "Leave this machine for tied
>>> jobs only"? I'd expect that setting for Master and Hadoop nodes, and
>>> let the others pick up any job.
>>
>> That would be preferable, but for legacy reasons Vesta and Minerva are
>> left for tied jobs. This was because the Master was the only build node
>> for 1.5+ years and had lots and lots of build on it when we then added
>> Vesta and Minerva. For compatibility reasons, we set it up as is.
>>
>> Suggestions on how to change this now? How to migrate builds off
>> Master? Clearly the extremes are "rip the band-aid off -- builds start
>> failing that try to run on Master" & "big project to contact build
>> owners and push them to migrate".
>
> Just tie jobs to master that have dependencies there, and mark it for
> tied jobs only, and let other jobs target labels if they have specific
> OS/CPU requirements.
>
> I don't think anything is particularly 'broken' at the moment is it? I
> was just trying to understand the current set-up, and if we ask new jobs
> to set up a bit differently we can prevent over burdening master while
> leaving spare capacity elsewhere.
good plan. that hadn't occurred to me ;) +1
It's at least a very good start.
--
--j.
Re: Hudson machine utilization
Posted by Tim Ellison <t....@gmail.com>.
On 14/Nov/2009 04:46, Nigel Daley wrote:
>>> I agree we should encourage folks to tie their linux builds to the
>>> "Ubuntu" label (which already exists), so both minerva and vesta get
>>> used.
>>>
>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>>
>> Why are minerva and vesta configured as "Leave this machine for tied
>> jobs only"? I'd expect that setting for Master and Hadoop nodes, and
>> let the others pick up any job.
>
> That would be preferable, but for legacy reasons Vesta and Minerva are
> left for tied jobs. This was because the Master was the only build node
> for 1.5+ years and had lots and lots of build on it when we then added
> Vesta and Minerva. For compatibility reasons, we set it up as is.
>
> Suggestions on how to change this now? How to migrate builds off
> Master? Clearly the extremes are "rip the band-aid off -- builds start
> failing that try to run on Master" & "big project to contact build
> owners and push them to migrate".
Just tie jobs to master that have dependencies there, and mark it for
tied jobs only, and let other jobs target labels if they have specific
OS/CPU requirements.
I don't think anything is particularly 'broken' at the moment is it? I
was just trying to understand the current set-up, and if we ask new jobs
to set up a bit differently we can prevent over burdening master while
leaving spare capacity elsewhere.
Regards,
Tim
Re: Hudson machine utilization
Posted by Nigel Daley <nd...@yahoo-inc.com>.
>> I agree we should encourage folks to tie their linux builds to the
>> "Ubuntu" label (which already exists), so both minerva and vesta
>> get used.
>>
>> We should also encourage projects (spam-assasin, ftpserver, struts,
>> vysper, xwork2) to move off of the Master hudson.zones.apache.org
>
> Why are minerva and vesta configured as "Leave this machine for tied
> jobs only"? I'd expect that setting for Master and Hadoop nodes, and
> let the others pick up any job.
That would be preferable, but for legacy reasons Vesta and Minerva are
left for tied jobs. This was because the Master was the only build
node for 1.5+ years and had lots and lots of build on it when we then
added Vesta and Minerva. For compatibility reasons, we set it up as is.
Suggestions on how to change this now? How to migrate builds off
Master? Clearly the extremes are "rip the band-aid off -- builds
start failing that try to run on Master" & "big project to contact
build owners and push them to migrate".
Nige
Re: Hudson machine utilization
Posted by Tim Ellison <t....@gmail.com>.
On 04/Nov/2009 23:18, Nigel Daley wrote:
> Tim, the Hadoop labeled machines were not donated to ASF. Minerva,
> Vesta, and a couple others (used now for buildbot) were donated to ASF.
Ok, that is fair enough.
> I agree we should encourage folks to tie their linux builds to the
> "Ubuntu" label (which already exists), so both minerva and vesta get used.
>
> We should also encourage projects (spam-assasin, ftpserver, struts,
> vysper, xwork2) to move off of the Master hudson.zones.apache.org
Why are minerva and vesta configured as "Leave this machine for tied
jobs only"? I'd expect that setting for Master and Hadoop nodes, and
let the others pick up any job.
Regards,
Tim
Re: Hudson machine utilization
Posted by Justin Mason <jm...@jmason.org>.
On Sat, Nov 14, 2009 at 04:47, Nigel Daley <nd...@yahoo-inc.com> wrote:
>
> On Nov 5, 2009, at 2:13 PM, Nigel Daley wrote:
>
>>
>> On Nov 5, 2009, at 2:10 PM, Tim Ellison wrote:
>>
>>> On 05/Nov/2009 12:48, Niklas Gustavsson wrote:
>>>>
>>>> On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org> wrote:
>>>>>
>>>>> We should also encourage projects (spam-assasin, ftpserver, struts,
>>>>> vysper,
>>>>> xwork2) to move off of the Master hudson.zones.apache.org
>>>>
>>>> As for FtpServer, we want our builds on Solaris (in addition to Linux
>>>> on which we also builds). Would it be beneficial to provide a Hudson
>>>> slave on a separate Solaris zone from where master is running?
>>>
>>> Yes, I think it would be preferable. Hudson is running on
>>> lucene.zones.apache.org but I suggest we ask infra for a dedicated
>>> Hudson zone rather than encourage individual projects to set up
>>> executors.
>>>
>>> WDYT?
>>
>> +1!
>
> Gavin, do you know how we request a new Solaris build slave for Hudson?
This would definitely be a "least pain" method of getting builds off the master.
Personally, I've been meaning to migrate the SA builds off, but the pain of
having to reinstall the CPAN dependencies required on whatever linux build
host we'd wind up using, was too great to get traction.
--
--j.
Re: Hudson machine utilization
Posted by Nigel Daley <nd...@yahoo-inc.com>.
On Nov 5, 2009, at 2:13 PM, Nigel Daley wrote:
>
> On Nov 5, 2009, at 2:10 PM, Tim Ellison wrote:
>
>> On 05/Nov/2009 12:48, Niklas Gustavsson wrote:
>>> On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org>
>>> wrote:
>>>> We should also encourage projects (spam-assasin, ftpserver,
>>>> struts, vysper,
>>>> xwork2) to move off of the Master hudson.zones.apache.org
>>>
>>> As for FtpServer, we want our builds on Solaris (in addition to
>>> Linux
>>> on which we also builds). Would it be beneficial to provide a Hudson
>>> slave on a separate Solaris zone from where master is running?
>>
>> Yes, I think it would be preferable. Hudson is running on
>> lucene.zones.apache.org but I suggest we ask infra for a dedicated
>> Hudson zone rather than encourage individual projects to set up
>> executors.
>>
>> WDYT?
>
> +1!
Gavin, do you know how we request a new Solaris build slave for Hudson?
Thx,
Nige
Re: Hudson machine utilization
Posted by Nigel Daley <nd...@yahoo-inc.com>.
On Nov 5, 2009, at 2:10 PM, Tim Ellison wrote:
> On 05/Nov/2009 12:48, Niklas Gustavsson wrote:
>> On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org>
>> wrote:
>>> We should also encourage projects (spam-assasin, ftpserver,
>>> struts, vysper,
>>> xwork2) to move off of the Master hudson.zones.apache.org
>>
>> As for FtpServer, we want our builds on Solaris (in addition to Linux
>> on which we also builds). Would it be beneficial to provide a Hudson
>> slave on a separate Solaris zone from where master is running?
>
> Yes, I think it would be preferable. Hudson is running on
> lucene.zones.apache.org but I suggest we ask infra for a dedicated
> Hudson zone rather than encourage individual projects to set up
> executors.
>
> WDYT?
+1!
Re: Hudson machine utilization
Posted by Tim Ellison <t....@gmail.com>.
On 05/Nov/2009 12:48, Niklas Gustavsson wrote:
> On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org> wrote:
>> We should also encourage projects (spam-assasin, ftpserver, struts, vysper,
>> xwork2) to move off of the Master hudson.zones.apache.org
>
> As for FtpServer, we want our builds on Solaris (in addition to Linux
> on which we also builds). Would it be beneficial to provide a Hudson
> slave on a separate Solaris zone from where master is running?
Yes, I think it would be preferable. Hudson is running on
lucene.zones.apache.org but I suggest we ask infra for a dedicated
Hudson zone rather than encourage individual projects to set up executors.
WDYT?
Regards,
Tim
Re: Hudson machine utilization
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org> wrote:
> We should also encourage projects (spam-assasin, ftpserver, struts, vysper,
> xwork2) to move off of the Master hudson.zones.apache.org
As for FtpServer, we want our builds on Solaris (in addition to Linux
on which we also builds). Would it be beneficial to provide a Hudson
slave on a separate Solaris zone from where master is running?
/niklas
Re: Hudson machine utilization
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Thu, Nov 5, 2009 at 12:18 AM, Nigel Daley <ni...@apache.org> wrote:
> I agree we should encourage folks to tie their linux builds to the "Ubuntu"
> label (which already exists), so both minerva and vesta get used.
Done for FtpServer and Vysper. Also moved our builds tied to "master"
to the "Solaris 10" group so we should be fine if an additional
Solaris zone is set up.
/niklas