You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Julian Foad <ju...@btopenworld.com> on 2015/01/07 16:03:33 UTC
lares_ubuntu buildbot slave - random failure in capturing stdout
Dear builds.at.a.o people,
The build slave http://ci.apache.org/buildslaves/lares_ubuntu
has spurious failures. This slave is used for RAT build testing for the Subversion project, and for several other builders.
The failure mode is it sometimes fails to see the stdout from programs it runs (such as 'svn info --xml' or 'git --version'). Examples can be seen in about 1 in 10 of the builds it attempts. Look at the "stdio log" of attempts that failed on step 1. Recent examples:
http://ci.apache.org/builders/svn-trunk-rat-report/builds/958
http://ci.apache.org/builders/log4j2-nightly/builds/210
http://ci.apache.org/builders/openjpa-1.2.x-docs/builds/201
http://ci.apache.org/builders/openjpa-1.0.x-docs/builds/197
An example of a broken attempt, from
http://ci.apache.org/builders/log4j2-nightly/builds/210/steps/git/logs/stdio
[[[
git --version
in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
watching logfiles {}
argv: ['git', '--version']
environment:
[...]
using PTY: True
program finished with exit code 0
elapsedTime=0.011104
]]]
An example of a working attempt (note the "git version 1.9.1" line), from
http://ci.apache.org/builders/log4j2-nightly/builds/211/steps/git/logs/stdio
[[[
git --version
in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
watching logfiles {}
argv: ['git', '--version']
environment:
[...]
using PTY: True
git version 1.9.1
program finished with exit code 0
elapsedTime=0.011104
]]]
This has been happening for weeks/months. For a long time we in the Subversion project were wondering if it was a bug in 'svn info --xml', but now I see it is not specific to Subversion and also affects other projects.
Please can someone look into it?
- Julian
Re: lares_ubuntu buildbot slave - random failure in capturing stdout
Posted by Julian Foad <ju...@btopenworld.com>.
I have filed a ticket so this doesn't get lost:
https://issues.apache.org/jira/browse/INFRA-8997
Since the false alerts on IRC bother me, I plan to disable Subversion's RAT test, in
svn.a.o/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/subversion.conf
soon, until it is fixed.
- Julian
I (Julian Foad) wrote:
> The build slave http://ci.apache.org/buildslaves/lares_ubuntu
> has spurious failures. This slave is used for RAT build testing for the
> Subversion project, and for several other builders.
[...]
Re: lares_ubuntu buildbot slave - random failure in capturing stdout
Posted by Julian Foad <ju...@btopenworld.com>.
I have filed a ticket so this doesn't get lost:
https://issues.apache.org/jira/browse/INFRA-8997
Since the false alerts on IRC bother me, I plan to disable Subversion's RAT test, in
svn.a.o/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/subversion.conf
soon, until it is fixed.
- Julian
I (Julian Foad) wrote:
> The build slave http://ci.apache.org/buildslaves/lares_ubuntu
> has spurious failures. This slave is used for RAT build testing for the
> Subversion project, and for several other builders.
[...]
Re: lares_ubuntu buildbot slave - random failure in capturing stdout
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Fuhrmann wrote on Thu, Jan 08, 2015 at 15:03:09 +0100:
> Hey Julian,
>
> Thank you so much for looking into this!
> Once this one gets fixed, there will be one less
> source of blood pressure spikes.
>
Isn't there a way to mark the build as purple (buildbot-level problem)
as opposed to red (project-level problem) when the 'git --version' or
'svn info --xml' step fails?
Daniel
> -- Stefan^2.
>
>
> On Wed, Jan 7, 2015 at 4:03 PM, Julian Foad <ju...@btopenworld.com>
> wrote:
>
> > Dear builds.at.a.o people,
> >
> > The build slave http://ci.apache.org/buildslaves/lares_ubuntu
> > has spurious failures. This slave is used for RAT build testing for the
> > Subversion project, and for several other builders.
> >
> > The failure mode is it sometimes fails to see the stdout from programs it
> > runs (such as 'svn info --xml' or 'git --version'). Examples can be seen in
> > about 1 in 10 of the builds it attempts. Look at the "stdio log" of
> > attempts that failed on step 1. Recent examples:
> >
> > http://ci.apache.org/builders/svn-trunk-rat-report/builds/958
> > http://ci.apache.org/builders/log4j2-nightly/builds/210
> > http://ci.apache.org/builders/openjpa-1.2.x-docs/builds/201
> > http://ci.apache.org/builders/openjpa-1.0.x-docs/builds/197
> >
> > An example of a broken attempt, from
> >
> > http://ci.apache.org/builders/log4j2-nightly/builds/210/steps/git/logs/stdio
> > [[[
> > git --version
> > in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> > watching logfiles {}
> > argv: ['git', '--version']
> > environment:
> > [...]
> > using PTY: True
> > program finished with exit code 0
> > elapsedTime=0.011104
> > ]]]
> >
> > An example of a working attempt (note the "git version 1.9.1" line), from
> >
> > http://ci.apache.org/builders/log4j2-nightly/builds/211/steps/git/logs/stdio
> > [[[
> > git --version
> > in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> > watching logfiles {}
> > argv: ['git', '--version']
> > environment:
> > [...]
> > using PTY: True
> > git version 1.9.1
> > program finished with exit code 0
> > elapsedTime=0.011104
> > ]]]
> >
> > This has been happening for weeks/months. For a long time we in the
> > Subversion project were wondering if it was a bug in 'svn info --xml', but
> > now I see it is not specific to Subversion and also affects other projects.
> >
> > Please can someone look into it?
> >
> > - Julian
> >
Re: lares_ubuntu buildbot slave - random failure in capturing stdout
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Stefan Fuhrmann wrote on Thu, Jan 08, 2015 at 15:03:09 +0100:
> Hey Julian,
>
> Thank you so much for looking into this!
> Once this one gets fixed, there will be one less
> source of blood pressure spikes.
>
Isn't there a way to mark the build as purple (buildbot-level problem)
as opposed to red (project-level problem) when the 'git --version' or
'svn info --xml' step fails?
Daniel
> -- Stefan^2.
>
>
> On Wed, Jan 7, 2015 at 4:03 PM, Julian Foad <ju...@btopenworld.com>
> wrote:
>
> > Dear builds.at.a.o people,
> >
> > The build slave http://ci.apache.org/buildslaves/lares_ubuntu
> > has spurious failures. This slave is used for RAT build testing for the
> > Subversion project, and for several other builders.
> >
> > The failure mode is it sometimes fails to see the stdout from programs it
> > runs (such as 'svn info --xml' or 'git --version'). Examples can be seen in
> > about 1 in 10 of the builds it attempts. Look at the "stdio log" of
> > attempts that failed on step 1. Recent examples:
> >
> > http://ci.apache.org/builders/svn-trunk-rat-report/builds/958
> > http://ci.apache.org/builders/log4j2-nightly/builds/210
> > http://ci.apache.org/builders/openjpa-1.2.x-docs/builds/201
> > http://ci.apache.org/builders/openjpa-1.0.x-docs/builds/197
> >
> > An example of a broken attempt, from
> >
> > http://ci.apache.org/builders/log4j2-nightly/builds/210/steps/git/logs/stdio
> > [[[
> > git --version
> > in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> > watching logfiles {}
> > argv: ['git', '--version']
> > environment:
> > [...]
> > using PTY: True
> > program finished with exit code 0
> > elapsedTime=0.011104
> > ]]]
> >
> > An example of a working attempt (note the "git version 1.9.1" line), from
> >
> > http://ci.apache.org/builders/log4j2-nightly/builds/211/steps/git/logs/stdio
> > [[[
> > git --version
> > in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> > watching logfiles {}
> > argv: ['git', '--version']
> > environment:
> > [...]
> > using PTY: True
> > git version 1.9.1
> > program finished with exit code 0
> > elapsedTime=0.011104
> > ]]]
> >
> > This has been happening for weeks/months. For a long time we in the
> > Subversion project were wondering if it was a bug in 'svn info --xml', but
> > now I see it is not specific to Subversion and also affects other projects.
> >
> > Please can someone look into it?
> >
> > - Julian
> >
Re: lares_ubuntu buildbot slave - random failure in capturing stdout
Posted by Stefan Fuhrmann <st...@wandisco.com>.
Hey Julian,
Thank you so much for looking into this!
Once this one gets fixed, there will be one less
source of blood pressure spikes.
-- Stefan^2.
On Wed, Jan 7, 2015 at 4:03 PM, Julian Foad <ju...@btopenworld.com>
wrote:
> Dear builds.at.a.o people,
>
> The build slave http://ci.apache.org/buildslaves/lares_ubuntu
> has spurious failures. This slave is used for RAT build testing for the
> Subversion project, and for several other builders.
>
> The failure mode is it sometimes fails to see the stdout from programs it
> runs (such as 'svn info --xml' or 'git --version'). Examples can be seen in
> about 1 in 10 of the builds it attempts. Look at the "stdio log" of
> attempts that failed on step 1. Recent examples:
>
> http://ci.apache.org/builders/svn-trunk-rat-report/builds/958
> http://ci.apache.org/builders/log4j2-nightly/builds/210
> http://ci.apache.org/builders/openjpa-1.2.x-docs/builds/201
> http://ci.apache.org/builders/openjpa-1.0.x-docs/builds/197
>
> An example of a broken attempt, from
>
> http://ci.apache.org/builders/log4j2-nightly/builds/210/steps/git/logs/stdio
> [[[
> git --version
> in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> watching logfiles {}
> argv: ['git', '--version']
> environment:
> [...]
> using PTY: True
> program finished with exit code 0
> elapsedTime=0.011104
> ]]]
>
> An example of a working attempt (note the "git version 1.9.1" line), from
>
> http://ci.apache.org/builders/log4j2-nightly/builds/211/steps/git/logs/stdio
> [[[
> git --version
> in dir /home/buildslave2/slave2/log4j2-nightly/build (timeout 1200 secs)
> watching logfiles {}
> argv: ['git', '--version']
> environment:
> [...]
> using PTY: True
> git version 1.9.1
> program finished with exit code 0
> elapsedTime=0.011104
> ]]]
>
> This has been happening for weeks/months. For a long time we in the
> Subversion project were wondering if it was a bug in 'svn info --xml', but
> now I see it is not specific to Subversion and also affects other projects.
>
> Please can someone look into it?
>
> - Julian
>