You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Uwe Schindler <uw...@thetaphi.de> on 2010/02/07 10:41:13 UTC
Hudson stuck on lucene-zones
The hudson build of lucene java is stuck since a few days. We cancelled the build two days ago, but Hudson stopped somehow during publishing the results. What can we do, we found no hanging process?
http://hudson.zones.apache.org/hudson/job/Lucene-trunk/1085/console
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
Re: Hudson stuck on lucene-zones
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Mon, Feb 8, 2010 at 11:58 PM, Jukka Zitting <ju...@gmail.com> wrote:
> Yep, I'll do the upgrade as soon as there's a good window between
> long-running build jobs.
Note that they also managed to break the update functionality, so you
will probably need to do a manual upgrade.
/niklas
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Feb 8, 2010 at 11:58 PM, Jukka Zitting <ju...@gmail.com> wrote:
> Yep, I'll do the upgrade as soon as there's a good window between
> long-running build jobs.
The upgrade is now done.
BR,
Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
On Mon, Feb 8, 2010 at 11:53 PM, Niklas Gustavsson <ni...@protocol7.com> wrote:
> On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Only: I cannot see anything in the HTML console log (only the rotating star). But the plain text log works.
>
> Seems like this was fixed in a follow-up release of Hudson:
> http://blog.hudson-ci.org/content/breaking-hudson-1345-released
Yep, I'll do the upgrade as soon as there's a good window between
long-running build jobs.
BR,
Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Niklas Gustavsson <ni...@protocol7.com>.
On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Only: I cannot see anything in the HTML console log (only the rotating star). But the plain text log works.
Seems like this was fixed in a follow-up release of Hudson:
http://blog.hudson-ci.org/content/breaking-hudson-1345-released
/niklas
Re: Hudson stuck on lucene-zones
Posted by Kristian Waagan <kr...@apache.org>.
Uwe Schindler wrote:
> Hi,
>
> Strange, I cannot see anything with any browser I use. It shows that some kilobytes of text were left out (but all text is left out). When I click on "show all" I only see the rotating star nothing more. And this happened only after the restart.
Hi,
I saw something similar before the upgrade and the restart.
A job I started manually, which ended up being queued, claimed to have
been running for more than 7 hours. The console output only showed the
rotating star. Then I killed another job that appeared stuck. This
brought the other job forwards, and I could see the console output.
However, the job took less than a minute to run due to an error in the
ant build file. How come it ran for more than 7 hours?
I believe the jobs ran on minerva, but to me it looked like some of the
other slaves were experiencing stuck jobs as well (jobs normally
completing within an hour or two were reported to have been running for
three to eight hours).
Regards,
--
Kristian
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
>> Sent: Sunday, February 07, 2010 2:44 PM
>> To: builds@apache.org
>> Subject: Re: Hudson stuck on lucene-zones
>>
>> Hi,
>>
>> On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
>>> Only: I cannot see anything in the HTML console log (only the
>> rotating star).
>>> But the plain text log works.
>> The HTML log works for me.
>>
>> BR,
>>
>> Jukka Zitting
>
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Mon, Feb 8, 2010 at 1:43 PM, sebb <se...@gmail.com> wrote:
> Ditto; just started a JMeter build, and the raw text looks fine, but
> the HTML console is showing the rotating logo only, no output, even if
> I do a force refresh. However, refresh does update the progress bar.
It looks like the console works fine on Google Chrome but not on
Firefox. I believe this issue was introduced when I upgraded the
Hudson version. See also:
http://issues.hudson-ci.org/browse/HUDSON-5540
Unless this is a big problem I'd keep the upgraded Hudson version for
now and do another upgrade as soon as a fix to HUDSON-5540 is
released. Otherwise I can downgrade Hudson to a previous version that
doesn't have this issue.
BR,
Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by sebb <se...@gmail.com>.
Ditto; just started a JMeter build, and the raw text looks fine, but
the HTML console is showing the rotating logo only, no output, even if
I do a force refresh. However, refresh does update the progress bar.
The raw text output does require a refresh, but that is to be expected.
After the job finished, I did a refresh of the HTML console, and the
output all appeared.
Maybe there is some kind of flush() command missing from the HTML?
On 07/02/2010, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hi,
>
> Strange, I cannot see anything with any browser I use. It shows that some kilobytes of text were left out (but all text is left out). When I click on "show all" I only see the rotating star nothing more. And this happened only after the restart.
>
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
>
> > Sent: Sunday, February 07, 2010 2:44 PM
> > To: builds@apache.org
> > Subject: Re: Hudson stuck on lucene-zones
> >
>
> > Hi,
> >
> > On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > > Only: I cannot see anything in the HTML console log (only the
> > rotating star).
> > > But the plain text log works.
> >
> > The HTML log works for me.
> >
> > BR,
> >
> > Jukka Zitting
>
>
RE: Hudson stuck on lucene-zones
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
Strange, I cannot see anything with any browser I use. It shows that some kilobytes of text were left out (but all text is left out). When I click on "show all" I only see the rotating star nothing more. And this happened only after the restart.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
> Sent: Sunday, February 07, 2010 2:44 PM
> To: builds@apache.org
> Subject: Re: Hudson stuck on lucene-zones
>
> Hi,
>
> On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > Only: I cannot see anything in the HTML console log (only the
> rotating star).
> > But the plain text log works.
>
> The HTML log works for me.
>
> BR,
>
> Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Feb 7, 2010 at 2:37 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Only: I cannot see anything in the HTML console log (only the rotating star).
> But the plain text log works.
The HTML log works for me.
BR,
Jukka Zitting
RE: Hudson stuck on lucene-zones
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Jukka,
it seems to work -- many, many thanks!
Only: I cannot see anything in the HTML console log (only the rotating star). But the plain text log works.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
> Sent: Sunday, February 07, 2010 2:31 PM
> To: builds@apache.org
> Subject: Re: Hudson stuck on lucene-zones
>
> Hi,
>
> On Sun, Feb 7, 2010 at 1:23 PM, Jukka Zitting <ju...@gmail.com>
> wrote:
> > I'll restart the entire Hudson instance as soon as all the remaining
> > active builds have been completed.
>
> Hudson is now restarted (and upgraded to latest version).
>
> I started a lucene-trunk build and at least for now it runs with no
> apparent problems.
>
> BR,
>
> Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Feb 7, 2010 at 1:23 PM, Jukka Zitting <ju...@gmail.com> wrote:
> I'll restart the entire Hudson instance as soon as all the remaining
> active builds have been completed.
Hudson is now restarted (and upgraded to latest version).
I started a lucene-trunk build and at least for now it runs with no
apparent problems.
BR,
Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Feb 7, 2010 at 1:06 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Weh ad already build running with an average load of 16, so 20 is a little
> bit higher (lol), but it should not stop completely.
It looks like we also have a few Hadoop builds that are mysteriously stuck.
I'll restart the entire Hudson instance as soon as all the remaining
active builds have been completed.
BR,
Jukka Zitting
RE: Hudson stuck on lucene-zones
Posted by Uwe Schindler <uw...@thetaphi.de>.
Weh ad already build running with an average load of 16, so 20 is a little bit higher (lol), but it should not stop completely.
Maybe I should open an issue, that somebody looks after the whole machine with all its zones.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
> Sent: Sunday, February 07, 2010 1:00 PM
> To: builds@apache.org
> Subject: Re: Hudson stuck on lucene-zones
>
> Hi,
>
> On Sun, Feb 7, 2010 at 12:33 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > Thanks! Now it seems that the next job (nutch) does nothing, the log
> only contains
> > one line but nothing is running. Somehow the whole Hudson JVM seems
> to behave
> > strange. Maybe we have filesystem full or something like that? I
> think we should
> > restart the build processor.
>
> I restarted the Hudson slave process, but this doesn't seem to do any
> good. There's plenty of disk space left and no runaway processes on
> lucene.zones.apache.org, but curiously "top" shows that the system
> load is at almost 20. Perhaps some other zone on the same machine is
> using up all the processing power?
>
> BR,
>
> Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Feb 7, 2010 at 12:33 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Thanks! Now it seems that the next job (nutch) does nothing, the log only contains
> one line but nothing is running. Somehow the whole Hudson JVM seems to behave
> strange. Maybe we have filesystem full or something like that? I think we should
> restart the build processor.
I restarted the Hudson slave process, but this doesn't seem to do any
good. There's plenty of disk space left and no runaway processes on
lucene.zones.apache.org, but curiously "top" shows that the system
load is at almost 20. Perhaps some other zone on the same machine is
using up all the processing power?
BR,
Jukka Zitting
RE: Hudson stuck on lucene-zones
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Jukka,
Thanks! Now it seems that the next job (nutch) does nothing, the log only contains one line but nothing is running. Somehow the whole Hudson JVM seems to behave strange. Maybe we have filesystem full or something like that? I think we should restart the build processor.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Jukka Zitting [mailto:jukka.zitting@gmail.com]
> Sent: Sunday, February 07, 2010 11:49 AM
> To: builds@apache.org
> Subject: Re: Hudson stuck on lucene-zones
>
> Hi,
>
> On Sun, Feb 7, 2010 at 10:41 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > The hudson build of lucene java is stuck since a few days. We
> cancelled the build
> > two days ago, but Hudson stopped somehow during publishing the
> results.
> > What can we do, we found no hanging process?
> >
> > http://hudson.zones.apache.org/hudson/job/Lucene-trunk/1085/console
>
> I killed the build.
>
> BR,
>
> Jukka Zitting
Re: Hudson stuck on lucene-zones
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sun, Feb 7, 2010 at 10:41 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> The hudson build of lucene java is stuck since a few days. We cancelled the build
> two days ago, but Hudson stopped somehow during publishing the results.
> What can we do, we found no hanging process?
>
> http://hudson.zones.apache.org/hudson/job/Lucene-trunk/1085/console
I killed the build.
BR,
Jukka Zitting