You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by "Gav..." <ga...@16degrees.com.au> on 2010/09/19 01:49:39 UTC

Builds that have been failing for a while

Hi All,

There seems to be a fair few builds that fail alot and nothing is being done
by the projects to fix them.
Hudson is supposed to be here to help you guys test for failures and you fix
them, that's the point.

https://hudson.apache.org/hudson/job/HBase-TRUNK/ as an example has been
failing constantly for 21 days - 
not picking on HBase specifically here and I haven't looked into the cause,
but really, junit tests failing
for over 3 weeks , it should be fixed by now. (So I have disabled it until
someone from the project looks at it)

If there are many projects like this using up resources that more attentive
projects could otherwise be using
then I think they should be removed after ONE WEEK of failing at the VERY
LATEST.

Please, projects, be attentive to your builds.

If you need help then please ASK, but know that as always the Hudson admins
are volunteers, so provide whatever info
you feel will be helpful. It is in the Hudson Admins interests and
ALLcprojects interests to clean up your builds
and ensure they are fixed so that they 'usually' pass rather than usually
fail., this will help in the smoother
operations of our Hudson systems.

Thanks.

Gav...



Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Fri, Sep 24, 2010 at 11:24 AM, Jukka Zitting <ju...@gmail.com> wrote:
> That shouldn't be too much of a burden, or is it? It doesn't tie up
> executors like some of the other failing builds.

It does tie up an SCMTrigger which is a resource that keeps failing
and does require administration (they will get stuck when slaves fail
and requires killing or they will keep a thread stuck forever). That
said, it is certainly not as resource intensive as running the full
build.

> Instead of the time limit, would it make more sense to only disable
> those jobs where >n of the last builds have failed?

Reasonable idea, let me play around with a script for that purpose and
get back with a new list to compare.

/niklas

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Fri, Sep 24, 2010 at 2:36 PM, Tammo van Lessen <tv...@gmail.com> wrote:
> Perhaps it might make sense to also notify the respective PMCs and
> advertise this mailinglist as I could imagine that some PMCs are not
> even aware of it.

We do ask those which gets access to Hudson to follow this list for
this exact purpose.

/niklas

Re: Builds that have been failing for a while

Posted by Tommaso Teofili <to...@gmail.com>.
2010/9/24 Tammo van Lessen <tv...@gmail.com>

>
> Perhaps it might make sense to also notify the respective PMCs and
> advertise this mailinglist as I could imagine that some PMCs are not
> even aware of it.
>
> Tammo
>
> --
> Tammo van Lessen - http://www.taval.de
>

+1
Regarding clerezza-site job I am taking a look at possible failing causes.
Regards,
Tommaso

Re: Builds that have been failing for a while

Posted by Tammo van Lessen <tv...@gmail.com>.
Hi,

On 24.09.2010 14:20, Niklas Gustavsson wrote:
> On Fri, Sep 24, 2010 at 11:44 AM, Gav... <ga...@16degrees.com.au> wrote:
>> I would suggest either a combination of both methods - perhaps time of 30
>> days .and.
>> the last 5 builds failed, or something like that?
> 
> That was my plan as well. Here's the list of jobs that has failed for
> more than one month and with more than 3 unsuccessful builds in a row:
> 
[...]

Perhaps it might make sense to also notify the respective PMCs and
advertise this mailinglist as I could imagine that some PMCs are not
even aware of it.

Tammo

-- 
Tammo van Lessen - http://www.taval.de

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Fri, Sep 24, 2010 at 11:44 AM, Gav... <ga...@16degrees.com.au> wrote:
> I would suggest either a combination of both methods - perhaps time of 30
> days .and.
> the last 5 builds failed, or something like that?

That was my plan as well. Here's the list of jobs that has failed for
more than one month and with more than 3 unsuccessful builds in a row:

Cayenne-doc                                        | 1 mo 21 days
   | 13
clerezza-site                                      | 3 mo 21 days
   | 7
Felix-WebConsole                                   | 1 mo 23 days
   | 7
Hadoop-20-Build                                    | 2 yr 3 mo
   | 15
Hadoop-Hdfs-21-Build                               | 5 mo 6 days
   | 15
Hadoop-Hdfs-trunk                                  | 5 mo 20 days
   | 40
Hadoop-Mapreduce-21-Build                          | 3 mo 25 days
   | 15
Hadoop-Mapreduce-trunk                             | 3 mo 25 days
   | 38
Hadoop-Mapreduce-trunk-Commit                      | 5 mo 20 days
   | 30
Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 16 days
   | 18
Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 12 days
   | 8
Hama-Patch-Admin                                   | 1 mo 10 days
   | 5
Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 21 days
   | 11
Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 21 days
   | 11
Hive-trunk-h0.18                                   | 1 mo 10 days
   | 17
Hive-trunk-h0.19                                   | 4 mo 20 days
   | 17
jspf-trunk                                         | 1 mo 10 days
   | 5
Mahout-Patch-Admin                                 | 1 yr 11 mo
   | 5
mailet-standard-trunk                              | 2 mo 4 days
   | 4
Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 29 days
   | 5
Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 16 days
   | 4
Nutch-trunk                                        | 2 mo 19 days
   | 40
Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 17 days
   | 23
Pig-Patch-h8.grid.sp2.yahoo.net                    | 5 mo 0 days
   | 33
Shiro                                              | 1 mo 13 days
   | 4
tapestry-5.0-freestyle                             | 6 mo 11 days
   | 5
Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 20 days         | 11

/niklas

RE: Builds that have been failing for a while

Posted by "Gav..." <ga...@16degrees.com.au>.

> -----Original Message-----
> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
> Sent: Friday, 24 September 2010 7:25 PM
> To: builds@apache.org
> Subject: Re: Builds that have been failing for a while
> 
> Hi,
> 
> On Fri, Sep 24, 2010 at 11:05 AM, Niklas Gustavsson
> <ni...@protocol7.com> wrote:
> > These three builds are set to be checking for updates on a periodic
> > basis (polling the SCM every hour) and when upstream dependencies are
> > built.
> 
> That shouldn't be too much of a burden, or is it? It doesn't tie up
> executors like some of the other failing builds.
> 
> I'm all for disabling builds that continuously keep failing, but in
> these cases only the last build has failed, and I totally expect the
> builds to go blue again as soon as someone gets around to touching the
> codebases.
> 
> Instead of the time limit, would it make more sense to only disable
> those jobs where >n of the last builds have failed?

Depends on the trigger frequency, last n builds could be used up in one day
by some
projects and take months to reach for others.

I would suggest either a combination of both methods - perhaps time of 30
days .and.
the last 5 builds failed, or something like that?

this is a new thing that needs doing, we can't have everyone replying saying
'oh yeah
please don't disable my build due to blah ...' . Lets find a sensible
setting and
stick to it. The aim is to get people to fix their builds or they will be
disabled
until they are fixed, simple.

Gav...

> 
> BR,
> 
> Jukka Zitting



Re: Builds that have been failing for a while

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Fri, Sep 24, 2010 at 11:05 AM, Niklas Gustavsson
<ni...@protocol7.com> wrote:
> These three builds are set to be checking for updates on a periodic
> basis (polling the SCM every hour) and when upstream dependencies are
> built.

That shouldn't be too much of a burden, or is it? It doesn't tie up
executors like some of the other failing builds.

I'm all for disabling builds that continuously keep failing, but in
these cases only the last build has failed, and I totally expect the
builds to go blue again as soon as someone gets around to touching the
codebases.

Instead of the time limit, would it make more sense to only disable
those jobs where >n of the last builds have failed?

BR,

Jukka Zitting

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Fri, Sep 24, 2010 at 10:48 AM, Jukka Zitting <ju...@gmail.com> wrote:
> On Fri, Sep 24, 2010 at 10:37 AM, Niklas Gustavsson
> <ni...@protocol7.com> wrote:
>> Jackrabbit-1.6                                     | 1 mo 12 days
>> Jackrabbit-classloader                             | 3 mo 10 days
>> Jackrabbit-ocm                                     | 3 mo 10 days
>
> These are builds that are configured to run only when there's a change
> in the related codebase, so even if they've been red for a long time,
> they don't really consume build resources. As soon as someone gets
> around to fixing the pending errors, I expect the CI build to start up
> again automatically to verify the fix.
>
> I suggest that we only disable *periodic* building of codebases that
> have been failing for a long time.

These three builds are set to be checking for updates on a periodic
basis (polling the SCM every hour) and when upstream dependencies are
built.

/niklas

Re: Builds that have been failing for a while

Posted by Jukka Zitting <ju...@gmail.com>.
Hi,

On Fri, Sep 24, 2010 at 10:37 AM, Niklas Gustavsson
<ni...@protocol7.com> wrote:
> Be warned, I'll run the script to disable build which been failing for
> more than 31 days on Sunday. This is the current list of such jobs:
> [...]
> Jackrabbit-1.6                                     | 1 mo 12 days
> Jackrabbit-classloader                             | 3 mo 10 days
> Jackrabbit-ocm                                     | 3 mo 10 days

These are builds that are configured to run only when there's a change
in the related codebase, so even if they've been red for a long time,
they don't really consume build resources. As soon as someone gets
around to fixing the pending errors, I expect the CI build to start up
again automatically to verify the fix.

I suggest that we only disable *periodic* building of codebases that
have been failing for a long time.

BR,

Jukka Zitting

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
Be warned, I'll run the script to disable build which been failing for
more than 31 days on Sunday. This is the current list of such jobs:

ActiveMQ-SysTest-5.3                               | 6 mo 25 days
AsyncWeb                                           | 3 mo 2 days
Cayenne-doc                                        | 1 mo 21 days
clerezza-site                                      | 3 mo 21 days
Empire-DB multios                                  | 1 mo 23 days
Felix-FileInstall                                  | 1 mo 21 days
Felix-Gogo                                         | 2 mo 11 days
Felix-WebConsole                                   | 1 mo 23 days
Hadoop-20-Build                                    | 2 yr 3 mo
Hadoop-Hdfs-21-Build                               | 5 mo 5 days
Hadoop-Hdfs-trunk                                  | 5 mo 20 days
Hadoop-Mapreduce-21-Build                          | 3 mo 25 days
Hadoop-Mapreduce-trunk                             | 3 mo 25 days
Hadoop-Mapreduce-trunk-Commit                      | 5 mo 20 days
Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 21 days
Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 16 days
Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 12 days
Hama-Patch                                         | 2 mo 28 days
Hama-Patch-Admin                                   | 1 mo 10 days
Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 21 days
Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 21 days
Hive-trunk-h0.18                                   | 1 mo 10 days
Hive-trunk-h0.19                                   | 4 mo 20 days
Jackrabbit-1.6                                     | 1 mo 12 days
Jackrabbit-classloader                             | 3 mo 10 days
Jackrabbit-ocm                                     | 3 mo 10 days
jspf-trunk                                         | 1 mo 10 days
Mahout-Patch-Admin                                 | 1 yr 11 mo
mailet-standard-trunk                              | 2 mo 4 days
Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 27 days
Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 29 days
Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 16 days
Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 23 days
Nutch-trunk                                        | 2 mo 19 days
org.apache.kato.eclipse                            | 1 yr 2 mo
Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 17 days
Pig-Patch-h8.grid.sp2.yahoo.net                    | 5 mo 0 days
ServiceMix-Plugins                                 | 2 mo 12 days
ServiceMix-Utils                                   | 2 mo 10 days
ServiceMix3                                        | 1 mo 3 days
Shiro                                              | 1 mo 13 days
struts-annotations                                 | 1 yr 1 mo
tapestry-5.0-freestyle                             | 6 mo 11 days
TestBuilds                                         | 1 yr 0 mo
Turbine Fulcrum                                    | 3 mo 21 days
Tuscany-1x                                         | 9 mo 6 days
Tuscany-run-plugin                                 | 3 mo 28 days
Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 6 days
Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 20 days

/niklas

Re: Builds that have been failing for a while

Posted by Ted Dunning <te...@gmail.com>.
They are just going to disable it, not delete it.  No cause for alarm.

Patch processing would be nice to have.

On Tue, Sep 21, 2010 at 5:48 AM, Robin Anil <ro...@gmail.com> wrote:

> *>Mahout-Patch-Admin                                 | 1 yr 11 mo*
> *
> *
> Wasnt this going to be taken down? Or were we going to enable patch
> testing.
> I don't recall where the discussion went the last time. Anyways, any work
> that needs to be done, should be done quickly or infra will take this down
> soon.
>

Fwd: Builds that have been failing for a while

Posted by Robin Anil <ro...@gmail.com>.
*>Mahout-Patch-Admin                                 | 1 yr 11 mo*
*
*
Wasnt this going to be taken down? Or were we going to enable patch testing.
I don't recall where the discussion went the last time. Anyways, any work
that needs to be done, should be done quickly or infra will take this down
soon.

Robin
*
*
---------- Forwarded message ----------
From: Niklas Gustavsson <ni...@protocol7.com>
Date: Mon, Sep 20, 2010 at 4:08 PM
Subject: Re: Builds that have been failing for a while
To: builds@apache.org


On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
<ni...@protocol7.com> wrote:
> On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au> wrote:
>> If there are many projects like this using up resources that more
attentive
>> projects could otherwise be using
>> then I think they should be removed after ONE WEEK of failing at the VERY
>> LATEST.
>
> +1, I'll look into scripting this. Don't think we need to remove them,
> but we should at least disable them.

Given a limit of one month, these are the jobs that would be disabled
(sorry for the formatting):
ActiveMQ-SysTest-5.3                               | 6 mo 21 days
Ant-Build-Matrix                                   | 1 mo 3 days
Ant_JDK_1.4_Test                                   | 1 mo 1 day
AsyncWeb                                           | 2 mo 28 days
buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
Camel                                              | 1 mo 10 days
Cayenne-doc                                        | 1 mo 17 days
clerezza-site                                      | 3 mo 17 days
Derby-trunk_clover                                 | 7 mo 24 days
Empire-DB multios                                  | 1 mo 19 days
Felix-FileInstall                                  | 1 mo 17 days
Felix-Gogo                                         | 2 mo 7 days
Felix-WebConsole                                   | 1 mo 19 days
Hadoop-20-Build                                    | 2 yr 3 mo
Hadoop-Hdfs-21-Build                               | 5 mo 2 days
Hadoop-Hdfs-trunk                                  | 5 mo 16 days
Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
Hadoop-Mapreduce-trunk                             | 3 mo 21 days
Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
Hama-Patch                                         | 2 mo 24 days
Hama-Patch-Admin                                   | 1 mo 6 days
Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
Hive-trunk-h0.18                                   | 1 mo 6 days
Hive-trunk-h0.19                                   | 4 mo 16 days
Jackrabbit-1.6                                     | 1 mo 8 days
Jackrabbit-classloader                             | 3 mo 6 days
Jackrabbit-ocm                                     | 3 mo 6 days
jspf-trunk                                         | 1 mo 6 days


* Mahout-Patch-Admin                                 | 1 yr 11 mo*

mailet-standard-trunk                              | 2 mo 0 days
Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
MINA-1.0.X                                         | 3 mo 10 days
MINA-1.1.X                                         | 3 mo 10 days
Nutch-trunk                                        | 2 mo 15 days
ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
ODE-trunk-maven2-jdk5                              | 1 mo 11 days
org.apache.kato.eclipse                            | 1 yr 2 mo
Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
ServiceMix-Plugins                                 | 2 mo 8 days
ServiceMix-Specs                                   | 1 mo 2 days
ServiceMix-Utils                                   | 2 mo 6 days
Shiro                                              | 1 mo 9 days
sling-contrib-1.5                                  | 6 mo 14 days
struts-annotations                                 | 1 yr 0 mo
tapestry-5.0-freestyle                             | 6 mo 7 days
TestBuilds                                         | 1 yr 0 mo
Turbine Fulcrum                                    | 3 mo 17 days
Tuscany-1x                                         | 9 mo 2 days
Tuscany-2x                                         | 1 mo 6 days
Tuscany-run-plugin                                 | 3 mo 25 days
Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days


If your project is in that list, please fix it. I will run the script
to automatically disable those jobs failing for more than one month in
a few days. Now, should your job get disabled, you can simply enable
it again after fixing the build problems. No configuration or history
will be lost.

/niklas

Re: Fwd: Builds that have been failing for a while

Posted by Carsten Ziegeler <cz...@apache.org>.
Felix Meschberger  wrote
> Hi,
> 
> Am 20.09.2010 14:26, schrieb Justin Edelson:
>> Looks like the sling-contrib build is going to be disabled soon... 
> 
> Thanks for bringing this up... I once looked after why those tests are
> failing, but failed to follow-up on.
> 
> Anyway, here is the reason:
> 
> [INFO] Checking legal files in:
> org.apache.sling.launchpad.contrib-testing-4-SNAPSHOT.war
> [HUDSON] ...
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Artifact does not contain any legal files:
> org.apache.sling.launchpad.contrib-testing-4-SNAPSHOT.war
> 
> This also occurrs on a local build. Maybe we should create the same
> integration test setup for contrib as we have for the main integration
> tests ?
> 
As a quick fix I added the missing files (Notice and licence) to the
project. The module now pass.
But I agree that we should use the same setup (the contrib testing
currently uses for example 2.0.7-SNAPSHOT of the maven launchpad plugin;
not sure if this is required)

Carsten
-- 
Carsten Ziegeler
cziegeler@apache.org

Re: Fwd: Builds that have been failing for a while

Posted by Felix Meschberger <fm...@gmail.com>.
Hi,

Am 20.09.2010 14:26, schrieb Justin Edelson:
> Looks like the sling-contrib build is going to be disabled soon... 

Thanks for bringing this up... I once looked after why those tests are
failing, but failed to follow-up on.

Anyway, here is the reason:

[INFO] Checking legal files in:
org.apache.sling.launchpad.contrib-testing-4-SNAPSHOT.war
[HUDSON] ...
[INFO]
------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO]
------------------------------------------------------------------------
[INFO] Artifact does not contain any legal files:
org.apache.sling.launchpad.contrib-testing-4-SNAPSHOT.war

This also occurrs on a local build. Maybe we should create the same
integration test setup for contrib as we have for the main integration
tests ?

Regards
Felix

> 
> 
> Begin forwarded message:
> 
>> From: "Gav..." <ga...@16degrees.com.au>
>> Date: September 20, 2010 7:34:33 AM EDT
>> To: <bu...@apache.org>
>> Subject: RE: Builds that have been failing for a while
>> Reply-To: builds@apache.org
>>
> 
>>
>>
>>> -----Original Message-----
>>> From: Niklas Gustavsson [mailto:niklas@protocol7.com]
>>> Sent: Monday, 20 September 2010 8:39 PM
>>> To: builds@apache.org
>>> Subject: Re: Builds that have been failing for a while
>>>
>>> On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
>>> <ni...@protocol7.com> wrote:
>>>> On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au>
>>> wrote:
>>>>> If there are many projects like this using up resources that more
>>> attentive
>>>>> projects could otherwise be using
>>>>> then I think they should be removed after ONE WEEK of failing at the
>>> VERY
>>>>> LATEST.
>>>>
>>>> +1, I'll look into scripting this. Don't think we need to remove
>>> them,
>>>> but we should at least disable them.
>>>
>>> Given a limit of one month, these are the jobs that would be disabled
>>> (sorry for the formatting):
>>> ActiveMQ-SysTest-5.3                               | 6 mo 21 days
>>> Ant-Build-Matrix                                   | 1 mo 3 days
>>> Ant_JDK_1.4_Test                                   | 1 mo 1 day
>>> AsyncWeb                                           | 2 mo 28 days
>>> buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
>>> Camel                                              | 1 mo 10 days
>>> Cayenne-doc                                        | 1 mo 17 days
>>> clerezza-site                                      | 3 mo 17 days
>>> Derby-trunk_clover                                 | 7 mo 24 days
>>> Empire-DB multios                                  | 1 mo 19 days
>>> Felix-FileInstall                                  | 1 mo 17 days
>>> Felix-Gogo                                         | 2 mo 7 days
>>> Felix-WebConsole                                   | 1 mo 19 days
>>> Hadoop-20-Build                                    | 2 yr 3 mo
>>> Hadoop-Hdfs-21-Build                               | 5 mo 2 days
>>> Hadoop-Hdfs-trunk                                  | 5 mo 16 days
>>> Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
>>> Hadoop-Mapreduce-trunk                             | 3 mo 21 days
>>> Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
>>> Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
>>> Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
>>> Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
>>> Hama-Patch                                         | 2 mo 24 days
>>> Hama-Patch-Admin                                   | 1 mo 6 days
>>> Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
>>> Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
>>> Hive-trunk-h0.18                                   | 1 mo 6 days
>>> Hive-trunk-h0.19                                   | 4 mo 16 days
>>> Jackrabbit-1.6                                     | 1 mo 8 days
>>> Jackrabbit-classloader                             | 3 mo 6 days
>>> Jackrabbit-ocm                                     | 3 mo 6 days
>>> jspf-trunk                                         | 1 mo 6 days
>>> Mahout-Patch-Admin                                 | 1 yr 11 mo
>>> mailet-standard-trunk                              | 2 mo 0 days
>>> Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
>>> Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
>>> Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
>>> Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
>>> MINA-1.0.X                                         | 3 mo 10 days
>>> MINA-1.1.X                                         | 3 mo 10 days
>>> Nutch-trunk                                        | 2 mo 15 days
>>> ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
>>> ODE-trunk-maven2-jdk5                              | 1 mo 11 days
>>> org.apache.kato.eclipse                            | 1 yr 2 mo
>>> Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
>>> Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
>>> ServiceMix-Plugins                                 | 2 mo 8 days
>>> ServiceMix-Specs                                   | 1 mo 2 days
>>> ServiceMix-Utils                                   | 2 mo 6 days
>>> Shiro                                              | 1 mo 9 days
>>> sling-contrib-1.5                                  | 6 mo 14 days
>>> struts-annotations                                 | 1 yr 0 mo
>>> tapestry-5.0-freestyle                             | 6 mo 7 days
>>> TestBuilds                                         | 1 yr 0 mo
>>> Turbine Fulcrum                                    | 3 mo 17 days
>>> Tuscany-1x                                         | 9 mo 2 days
>>> Tuscany-2x                                         | 1 mo 6 days
>>> Tuscany-run-plugin                                 | 3 mo 25 days
>>> Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
>>> Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
>>>
>>>
>>> If your project is in that list, please fix it. I will run the script
>>> to automatically disable those jobs failing for more than one month in
>>> a few days. Now, should your job get disabled, you can simply enable
>>> it again after fixing the build problems. No configuration or history
>>> will be lost.
>>
>> +1 , nice work.
>>
>> Gav...
>>
>>>
>>> /niklas
>>
>>
> 

Fwd: Builds that have been failing for a while

Posted by Justin Edelson <ju...@gmail.com>.
Looks like the sling-contrib build is going to be disabled soon... 


Begin forwarded message:

> From: "Gav..." <ga...@16degrees.com.au>
> Date: September 20, 2010 7:34:33 AM EDT
> To: <bu...@apache.org>
> Subject: RE: Builds that have been failing for a while
> Reply-To: builds@apache.org
> 

> 
> 
>> -----Original Message-----
>> From: Niklas Gustavsson [mailto:niklas@protocol7.com]
>> Sent: Monday, 20 September 2010 8:39 PM
>> To: builds@apache.org
>> Subject: Re: Builds that have been failing for a while
>> 
>> On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
>> <ni...@protocol7.com> wrote:
>>> On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au>
>> wrote:
>>>> If there are many projects like this using up resources that more
>> attentive
>>>> projects could otherwise be using
>>>> then I think they should be removed after ONE WEEK of failing at the
>> VERY
>>>> LATEST.
>>> 
>>> +1, I'll look into scripting this. Don't think we need to remove
>> them,
>>> but we should at least disable them.
>> 
>> Given a limit of one month, these are the jobs that would be disabled
>> (sorry for the formatting):
>> ActiveMQ-SysTest-5.3                               | 6 mo 21 days
>> Ant-Build-Matrix                                   | 1 mo 3 days
>> Ant_JDK_1.4_Test                                   | 1 mo 1 day
>> AsyncWeb                                           | 2 mo 28 days
>> buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
>> Camel                                              | 1 mo 10 days
>> Cayenne-doc                                        | 1 mo 17 days
>> clerezza-site                                      | 3 mo 17 days
>> Derby-trunk_clover                                 | 7 mo 24 days
>> Empire-DB multios                                  | 1 mo 19 days
>> Felix-FileInstall                                  | 1 mo 17 days
>> Felix-Gogo                                         | 2 mo 7 days
>> Felix-WebConsole                                   | 1 mo 19 days
>> Hadoop-20-Build                                    | 2 yr 3 mo
>> Hadoop-Hdfs-21-Build                               | 5 mo 2 days
>> Hadoop-Hdfs-trunk                                  | 5 mo 16 days
>> Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
>> Hadoop-Mapreduce-trunk                             | 3 mo 21 days
>> Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
>> Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
>> Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
>> Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
>> Hama-Patch                                         | 2 mo 24 days
>> Hama-Patch-Admin                                   | 1 mo 6 days
>> Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
>> Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
>> Hive-trunk-h0.18                                   | 1 mo 6 days
>> Hive-trunk-h0.19                                   | 4 mo 16 days
>> Jackrabbit-1.6                                     | 1 mo 8 days
>> Jackrabbit-classloader                             | 3 mo 6 days
>> Jackrabbit-ocm                                     | 3 mo 6 days
>> jspf-trunk                                         | 1 mo 6 days
>> Mahout-Patch-Admin                                 | 1 yr 11 mo
>> mailet-standard-trunk                              | 2 mo 0 days
>> Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
>> Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
>> Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
>> Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
>> MINA-1.0.X                                         | 3 mo 10 days
>> MINA-1.1.X                                         | 3 mo 10 days
>> Nutch-trunk                                        | 2 mo 15 days
>> ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
>> ODE-trunk-maven2-jdk5                              | 1 mo 11 days
>> org.apache.kato.eclipse                            | 1 yr 2 mo
>> Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
>> Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
>> ServiceMix-Plugins                                 | 2 mo 8 days
>> ServiceMix-Specs                                   | 1 mo 2 days
>> ServiceMix-Utils                                   | 2 mo 6 days
>> Shiro                                              | 1 mo 9 days
>> sling-contrib-1.5                                  | 6 mo 14 days
>> struts-annotations                                 | 1 yr 0 mo
>> tapestry-5.0-freestyle                             | 6 mo 7 days
>> TestBuilds                                         | 1 yr 0 mo
>> Turbine Fulcrum                                    | 3 mo 17 days
>> Tuscany-1x                                         | 9 mo 2 days
>> Tuscany-2x                                         | 1 mo 6 days
>> Tuscany-run-plugin                                 | 3 mo 25 days
>> Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
>> Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
>> 
>> 
>> If your project is in that list, please fix it. I will run the script
>> to automatically disable those jobs failing for more than one month in
>> a few days. Now, should your job get disabled, you can simply enable
>> it again after fixing the build problems. No configuration or history
>> will be lost.
> 
> +1 , nice work.
> 
> Gav...
> 
>> 
>> /niklas
> 
> 

RE: Builds that have been failing for a while

Posted by "Gav..." <ga...@16degrees.com.au>.

> -----Original Message-----
> From: Niklas Gustavsson [mailto:niklas@protocol7.com]
> Sent: Monday, 20 September 2010 8:39 PM
> To: builds@apache.org
> Subject: Re: Builds that have been failing for a while
> 
> On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
> <ni...@protocol7.com> wrote:
> > On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au>
> wrote:
> >> If there are many projects like this using up resources that more
> attentive
> >> projects could otherwise be using
> >> then I think they should be removed after ONE WEEK of failing at the
> VERY
> >> LATEST.
> >
> > +1, I'll look into scripting this. Don't think we need to remove
> them,
> > but we should at least disable them.
> 
> Given a limit of one month, these are the jobs that would be disabled
> (sorry for the formatting):
> ActiveMQ-SysTest-5.3                               | 6 mo 21 days
> Ant-Build-Matrix                                   | 1 mo 3 days
> Ant_JDK_1.4_Test                                   | 1 mo 1 day
> AsyncWeb                                           | 2 mo 28 days
> buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
> Camel                                              | 1 mo 10 days
> Cayenne-doc                                        | 1 mo 17 days
> clerezza-site                                      | 3 mo 17 days
> Derby-trunk_clover                                 | 7 mo 24 days
> Empire-DB multios                                  | 1 mo 19 days
> Felix-FileInstall                                  | 1 mo 17 days
> Felix-Gogo                                         | 2 mo 7 days
> Felix-WebConsole                                   | 1 mo 19 days
> Hadoop-20-Build                                    | 2 yr 3 mo
> Hadoop-Hdfs-21-Build                               | 5 mo 2 days
> Hadoop-Hdfs-trunk                                  | 5 mo 16 days
> Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
> Hadoop-Mapreduce-trunk                             | 3 mo 21 days
> Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
> Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
> Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
> Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
> Hama-Patch                                         | 2 mo 24 days
> Hama-Patch-Admin                                   | 1 mo 6 days
> Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
> Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
> Hive-trunk-h0.18                                   | 1 mo 6 days
> Hive-trunk-h0.19                                   | 4 mo 16 days
> Jackrabbit-1.6                                     | 1 mo 8 days
> Jackrabbit-classloader                             | 3 mo 6 days
> Jackrabbit-ocm                                     | 3 mo 6 days
> jspf-trunk                                         | 1 mo 6 days
> Mahout-Patch-Admin                                 | 1 yr 11 mo
> mailet-standard-trunk                              | 2 mo 0 days
> Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
> Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
> Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
> Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
> MINA-1.0.X                                         | 3 mo 10 days
> MINA-1.1.X                                         | 3 mo 10 days
> Nutch-trunk                                        | 2 mo 15 days
> ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
> ODE-trunk-maven2-jdk5                              | 1 mo 11 days
> org.apache.kato.eclipse                            | 1 yr 2 mo
> Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
> Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
> ServiceMix-Plugins                                 | 2 mo 8 days
> ServiceMix-Specs                                   | 1 mo 2 days
> ServiceMix-Utils                                   | 2 mo 6 days
> Shiro                                              | 1 mo 9 days
> sling-contrib-1.5                                  | 6 mo 14 days
> struts-annotations                                 | 1 yr 0 mo
> tapestry-5.0-freestyle                             | 6 mo 7 days
> TestBuilds                                         | 1 yr 0 mo
> Turbine Fulcrum                                    | 3 mo 17 days
> Tuscany-1x                                         | 9 mo 2 days
> Tuscany-2x                                         | 1 mo 6 days
> Tuscany-run-plugin                                 | 3 mo 25 days
> Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
> Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
> 
> 
> If your project is in that list, please fix it. I will run the script
> to automatically disable those jobs failing for more than one month in
> a few days. Now, should your job get disabled, you can simply enable
> it again after fixing the build problems. No configuration or history
> will be lost.

+1 , nice work.

Gav...

> 
> /niklas



Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Mon, Sep 20, 2010 at 8:43 PM, Patrick Hunt <ph...@cloudera.com> wrote:
> Hi. Improving resource use is a great goal, I'm not sure it's that
> clearcut though. I'm only familiar with ZK: note that these two jobs
> are our patch queues, which only gets run when a user submits a patch
> to a jira (only a few patches on each job over the last couple
> months):
> Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
> Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
> this may fail for any number of reasons (patch won't apply, no tests,
> findbug issues, etc...) Also notice that a patch gets sent to only 1
> of 3 possible machines in some pseudo random fashion. So while one
> patch job shows a recent success, the others do not. So to some extent
> this is out of our hands.

I agree that this might be a problem. However, looking at these
specific jobs they both seem be out of the list if they had been fully
maintained. E.g. the following build seems to have failed due to a
build configuration problem:
https://hudson.apache.org/hudson/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/110/console

There a quite a few builds failing for the same reason. Should these
have been fixed, this job would not have been caught by the script.

That said, I'll be happy to maintain a white list of jobs in the
disable script. So, if you got a job that has a good reason for being
failed for a long time, let me know.

> We also see frequent failures from things that seem like infrastruture
> issues, here's there console output from a couple recent failures:
> WARNING: clock of the subversion server appears to be out of sync.
> This can result in inconsistent check out behavior.
>
> here's another:
> Checking out http://svn.apache.org/repos/asf/hadoop/zookeeper/trunk
> ERROR: Failed to check out
> http://svn.apache.org/repos/asf/hadoop/zookeeper/trunk
> org.tmatesoft.svn.core.SVNException: svn: unknown host
> svn: OPTIONS request failed on '/repos/asf/hadoop/zookeeper/trunk'
>
> that said, we recently had issues with our trunk that were causing
> intermittent failures. We've been working on those and hopefully it
> will help to clear these patch issues.

Yes, there will always be builds failing for these reasons (e.g.
Hudson instability, network problems). I would recommend to delete
these builds as they do not reflect problems in your build and doesn't
add much knowledge (besides bad statistics). That's what I do for the
jobs I maintain.

/niklas

Re: Builds that have been failing for a while

Posted by Patrick Hunt <ph...@cloudera.com>.
Hi. Improving resource use is a great goal, I'm not sure it's that
clearcut though. I'm only familiar with ZK: note that these two jobs
are our patch queues, which only gets run when a user submits a patch
to a jira (only a few patches on each job over the last couple
months):
Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
this may fail for any number of reasons (patch won't apply, no tests,
findbug issues, etc...) Also notice that a patch gets sent to only 1
of 3 possible machines in some pseudo random fashion. So while one
patch job shows a recent success, the others do not. So to some extent
this is out of our hands.

We also see frequent failures from things that seem like infrastruture
issues, here's there console output from a couple recent failures:
WARNING: clock of the subversion server appears to be out of sync.
This can result in inconsistent check out behavior.

here's another:
Checking out http://svn.apache.org/repos/asf/hadoop/zookeeper/trunk
ERROR: Failed to check out
http://svn.apache.org/repos/asf/hadoop/zookeeper/trunk
org.tmatesoft.svn.core.SVNException: svn: unknown host
svn: OPTIONS request failed on '/repos/asf/hadoop/zookeeper/trunk'

that said, we recently had issues with our trunk that were causing
intermittent failures. We've been working on those and hopefully it
will help to clear these patch issues.

Patrick


On Mon, Sep 20, 2010 at 7:22 AM, Niklas Gustavsson <ni...@protocol7.com> wrote:
>
> On Mon, Sep 20, 2010 at 2:10 PM, Kristian Waagan
> <kr...@oracle.com> wrote:
> > Just want to point out that not all the jobs on the list have been running
> > regularly, so they haven't been using resources.
> > In any case, disabling jobs that haven't been run for a long time is
> > probably ok too.
> > One could consider removing "dead jobs", but I want to keep the Derby job a
> > little longer ;)
> > (is failing because Clover is/was unable to handle the data volume, but
> > hasn't been run for nine months)
>
> Right, if the last build was successful, the job will not be disabled,
> even if the last build was older than one month. However, if your last
> job was unsuccessful and no one has fixed that for more than a month,
> I think disabling is probably appropriate. In your case, simply enable
> the job again when you fixed the problem with Clover. If you haven't
> used the job for nine months, unchecking a checkbox is probably not
> that much work when you're ready to run again :-)
>
> /niklas

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Mon, Sep 20, 2010 at 2:10 PM, Kristian Waagan
<kr...@oracle.com> wrote:
> Just want to point out that not all the jobs on the list have been running
> regularly, so they haven't been using resources.
> In any case, disabling jobs that haven't been run for a long time is
> probably ok too.
> One could consider removing "dead jobs", but I want to keep the Derby job a
> little longer ;)
> (is failing because Clover is/was unable to handle the data volume, but
> hasn't been run for nine months)

Right, if the last build was successful, the job will not be disabled,
even if the last build was older than one month. However, if your last
job was unsuccessful and no one has fixed that for more than a month,
I think disabling is probably appropriate. In your case, simply enable
the job again when you fixed the problem with Clover. If you haven't
used the job for nine months, unchecking a checkbox is probably not
that much work when you're ready to run again :-)

/niklas

Re: Builds that have been failing for a while

Posted by Kristian Waagan <kr...@oracle.com>.
  On 20.09.10 12:38, Niklas Gustavsson wrote:
> On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
> <ni...@protocol7.com>  wrote:
>> On Sun, Sep 19, 2010 at 1:49 AM, Gav...<ga...@16degrees.com.au>  wrote:
>>> If there are many projects like this using up resources that more attentive
>>> projects could otherwise be using
>>> then I think they should be removed after ONE WEEK of failing at the VERY
>>> LATEST.
>> +1, I'll look into scripting this. Don't think we need to remove them,
>> but we should at least disable them.
> Given a limit of one month, these are the jobs that would be disabled
> (sorry for the formatting):
> ActiveMQ-SysTest-5.3                               | 6 mo 21 days
> Ant-Build-Matrix                                   | 1 mo 3 days
> Ant_JDK_1.4_Test                                   | 1 mo 1 day
> AsyncWeb                                           | 2 mo 28 days
> buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
> Camel                                              | 1 mo 10 days
> Cayenne-doc                                        | 1 mo 17 days
> clerezza-site                                      | 3 mo 17 days
> Derby-trunk_clover                                 | 7 mo 24 days
> Empire-DB multios                                  | 1 mo 19 days
> Felix-FileInstall                                  | 1 mo 17 days
> Felix-Gogo                                         | 2 mo 7 days
> Felix-WebConsole                                   | 1 mo 19 days
> Hadoop-20-Build                                    | 2 yr 3 mo
> Hadoop-Hdfs-21-Build                               | 5 mo 2 days
> Hadoop-Hdfs-trunk                                  | 5 mo 16 days
> Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
> Hadoop-Mapreduce-trunk                             | 3 mo 21 days
> Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
> Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
> Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
> Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
> Hama-Patch                                         | 2 mo 24 days
> Hama-Patch-Admin                                   | 1 mo 6 days
> Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
> Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
> Hive-trunk-h0.18                                   | 1 mo 6 days
> Hive-trunk-h0.19                                   | 4 mo 16 days
> Jackrabbit-1.6                                     | 1 mo 8 days
> Jackrabbit-classloader                             | 3 mo 6 days
> Jackrabbit-ocm                                     | 3 mo 6 days
> jspf-trunk                                         | 1 mo 6 days
> Mahout-Patch-Admin                                 | 1 yr 11 mo
> mailet-standard-trunk                              | 2 mo 0 days
> Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
> Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
> Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
> Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
> MINA-1.0.X                                         | 3 mo 10 days
> MINA-1.1.X                                         | 3 mo 10 days
> Nutch-trunk                                        | 2 mo 15 days
> ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
> ODE-trunk-maven2-jdk5                              | 1 mo 11 days
> org.apache.kato.eclipse                            | 1 yr 2 mo
> Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
> Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
> ServiceMix-Plugins                                 | 2 mo 8 days
> ServiceMix-Specs                                   | 1 mo 2 days
> ServiceMix-Utils                                   | 2 mo 6 days
> Shiro                                              | 1 mo 9 days
> sling-contrib-1.5                                  | 6 mo 14 days
> struts-annotations                                 | 1 yr 0 mo
> tapestry-5.0-freestyle                             | 6 mo 7 days
> TestBuilds                                         | 1 yr 0 mo
> Turbine Fulcrum                                    | 3 mo 17 days
> Tuscany-1x                                         | 9 mo 2 days
> Tuscany-2x                                         | 1 mo 6 days
> Tuscany-run-plugin                                 | 3 mo 25 days
> Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
> Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days
>
>
> If your project is in that list, please fix it. I will run the script
> to automatically disable those jobs failing for more than one month in
> a few days. Now, should your job get disabled, you can simply enable
> it again after fixing the build problems. No configuration or history
> will be lost.

Hi,

I think it is a good idea to disable jobs that keep failing for a long 
time, +1. Thanks, Niklas.

Just want to point out that not all the jobs on the list have been 
running regularly, so they haven't been using resources.
In any case, disabling jobs that haven't been run for a long time is 
probably ok too.
One could consider removing "dead jobs", but I want to keep the Derby 
job a little longer ;)
(is failing because Clover is/was unable to handle the data volume, but 
hasn't been run for nine months)


-- 
Kristian

> /niklas


Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Sun, Sep 19, 2010 at 10:22 AM, Niklas Gustavsson
<ni...@protocol7.com> wrote:
> On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au> wrote:
>> If there are many projects like this using up resources that more attentive
>> projects could otherwise be using
>> then I think they should be removed after ONE WEEK of failing at the VERY
>> LATEST.
>
> +1, I'll look into scripting this. Don't think we need to remove them,
> but we should at least disable them.

Given a limit of one month, these are the jobs that would be disabled
(sorry for the formatting):
ActiveMQ-SysTest-5.3                               | 6 mo 21 days
Ant-Build-Matrix                                   | 1 mo 3 days
Ant_JDK_1.4_Test                                   | 1 mo 1 day
AsyncWeb                                           | 2 mo 28 days
buildr-ci-build-jruby-jdk5                         | 1 mo 18 days
Camel                                              | 1 mo 10 days
Cayenne-doc                                        | 1 mo 17 days
clerezza-site                                      | 3 mo 17 days
Derby-trunk_clover                                 | 7 mo 24 days
Empire-DB multios                                  | 1 mo 19 days
Felix-FileInstall                                  | 1 mo 17 days
Felix-Gogo                                         | 2 mo 7 days
Felix-WebConsole                                   | 1 mo 19 days
Hadoop-20-Build                                    | 2 yr 3 mo
Hadoop-Hdfs-21-Build                               | 5 mo 2 days
Hadoop-Hdfs-trunk                                  | 5 mo 16 days
Hadoop-Mapreduce-21-Build                          | 3 mo 21 days
Hadoop-Mapreduce-trunk                             | 3 mo 21 days
Hadoop-Mapreduce-trunk-Commit                      | 5 mo 16 days
Hadoop-Patch-h1.grid.sp2.yahoo.net                 | 3 mo 17 days
Hadoop-Patch-h4.grid.sp2.yahoo.net                 | 3 mo 12 days
Hadoop-Patch-h9.grid.sp2.yahoo.net                 | 8 mo 8 days
Hama-Patch                                         | 2 mo 24 days
Hama-Patch-Admin                                   | 1 mo 6 days
Hdfs-Patch-h2.grid.sp2.yahoo.net                   | 5 mo 17 days
Hdfs-Patch-h5.grid.sp2.yahoo.net                   | 5 mo 17 days
Hive-trunk-h0.18                                   | 1 mo 6 days
Hive-trunk-h0.19                                   | 4 mo 16 days
Jackrabbit-1.6                                     | 1 mo 8 days
Jackrabbit-classloader                             | 3 mo 6 days
Jackrabbit-ocm                                     | 3 mo 6 days
jspf-trunk                                         | 1 mo 6 days
Mahout-Patch-Admin                                 | 1 yr 11 mo
mailet-standard-trunk                              | 2 mo 0 days
Mapreduce-Patch-h3.grid.sp2.yahoo.net              | 4 mo 23 days
Mapreduce-Patch-h4.grid.sp2.yahoo.net              | 3 mo 25 days
Mapreduce-Patch-h6.grid.sp2.yahoo.net              | 4 mo 12 days
Mapreduce-Patch-h9.grid.sp2.yahoo.net              | 6 mo 20 days
MINA-1.0.X                                         | 3 mo 10 days
MINA-1.1.X                                         | 3 mo 10 days
Nutch-trunk                                        | 2 mo 15 days
ODE-trunk-m2-jdk5-nightly-deploy                   | 1 mo 2 days
ODE-trunk-maven2-jdk5                              | 1 mo 11 days
org.apache.kato.eclipse                            | 1 yr 2 mo
Pig-Patch-h7.grid.sp2.yahoo.net                    | 3 mo 13 days
Pig-Patch-h8.grid.sp2.yahoo.net                    | 4 mo 26 days
ServiceMix-Plugins                                 | 2 mo 8 days
ServiceMix-Specs                                   | 1 mo 2 days
ServiceMix-Utils                                   | 2 mo 6 days
Shiro                                              | 1 mo 9 days
sling-contrib-1.5                                  | 6 mo 14 days
struts-annotations                                 | 1 yr 0 mo
tapestry-5.0-freestyle                             | 6 mo 7 days
TestBuilds                                         | 1 yr 0 mo
Turbine Fulcrum                                    | 3 mo 17 days
Tuscany-1x                                         | 9 mo 2 days
Tuscany-2x                                         | 1 mo 6 days
Tuscany-run-plugin                                 | 3 mo 25 days
Zookeeper-Patch-h1.grid.sp2.yahoo.net              | 2 mo 2 days
Zookeeper-Patch-h7.grid.sp2.yahoo.net              | 1 mo 16 days


If your project is in that list, please fix it. I will run the script
to automatically disable those jobs failing for more than one month in
a few days. Now, should your job get disabled, you can simply enable
it again after fixing the build problems. No configuration or history
will be lost.

/niklas

Re: Builds that have been failing for a while

Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Sun, Sep 19, 2010 at 1:49 AM, Gav... <ga...@16degrees.com.au> wrote:
> If there are many projects like this using up resources that more attentive
> projects could otherwise be using
> then I think they should be removed after ONE WEEK of failing at the VERY
> LATEST.

+1, I'll look into scripting this. Don't think we need to remove them,
but we should at least disable them.

/niklas