You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Mike Jumper <mj...@apache.org> on 2020/01/14 06:11:38 UTC
Jenkins queue blockage
Hi all,
Is there a known issue currently affecting Jenkins? The build queue at the
moment stands at 419 jobs and growing, yet it doesn't look like there are
enough things running to account for the blockage.
Checking the load statistics graph for Jenkins, it looks like things
suddenly took a turn at around 23:00 UTC, with queue size increasing
despite available executors going up and busy executors going down:
https://builds.apache.org/load-statistics
Thanks,
- Mike
Re: Jenkins queue blockage
Posted by Michael Shuler <mi...@pbandjelly.org>.
The top level job is a lightweight executor run that triggers the actual
test job matrix, such as:
https://builds.apache.org/job/Cassandra-3.11-cqlsh-tests/511/cython=yes,jdk=JDK%201.8%20(latest),label=cassandra/
The lightweight executor can run anywhere, it is undefined, and it does
not take up an actual executor slot or do anything but sit there and
wait while the real jobs run tests on label=cassandra instances.
Kind regards,
Michael
On 1/14/20 8:07 AM, Zoran Regvart wrote:
> Hi Gavin,
> is this enforced somehow? I can see Cassandra-3.11-cqlsh-tests[1] running
> on the master node.
>
> zoran
>
> [1] https://builds.apache.org/job/Cassandra-3.11-cqlsh-tests/
>
> On Tue, Jan 14, 2020 at 12:44 PM Gavin McDonald <ip...@gmail.com> wrote:
>
>> Hi All,
>>
>> Jenkins is getting rebooted ...
>>
>> Note that no jobs 'tie to master' but pipeline jobs do get associated with
>> the master, no jobs get build there.
>>
>> Gav...
>>
>>
>> On Tue, Jan 14, 2020 at 11:37 AM Paul King <pa...@apache.org> wrote:
>>
>>>
>>> Am I reading this correctly?
>>> Nothing built on websites agent for 12 hrs:
>>> https://builds.apache.org/computer/websites/builds
>>>
>>> On 2020/01/14 11:24:44, Zoran Regvart <zo...@regvart.com> wrote:
>>>> Hi builders,
>>>> could it have something to do with some builds tied to the `master`
>>> node[1]?
>>>>
>>>> I thought it was a bad practice to tie the executors on `master`
>> node...
>>>>
>>>> zoran
>>>>
>>>> [1] https://builds.apache.org/computer/(master)/builds
>>>>
>>>> On Tue, Jan 14, 2020 at 7:12 AM Mike Jumper <mj...@apache.org>
>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Is there a known issue currently affecting Jenkins? The build queue
>> at
>>> the
>>>>> moment stands at 419 jobs and growing, yet it doesn't look like there
>>> are
>>>>> enough things running to account for the blockage.
>>>>>
>>>>> Checking the load statistics graph for Jenkins, it looks like things
>>>>> suddenly took a turn at around 23:00 UTC, with queue size increasing
>>>>> despite available executors going up and busy executors going down:
>>>>>
>>>>> https://builds.apache.org/load-statistics
>>>>>
>>>>> Thanks,
>>>>>
>>>>> - Mike
>>>>>
>>>>
>>>>
>>>> --
>>>> Zoran Regvart
>>>>
>>>
>>
>>
>> --
>> Gav...
>>
>
>
Re: Jenkins queue blockage
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Gavin,
is this enforced somehow? I can see Cassandra-3.11-cqlsh-tests[1] running
on the master node.
zoran
[1] https://builds.apache.org/job/Cassandra-3.11-cqlsh-tests/
On Tue, Jan 14, 2020 at 12:44 PM Gavin McDonald <ip...@gmail.com> wrote:
> Hi All,
>
> Jenkins is getting rebooted ...
>
> Note that no jobs 'tie to master' but pipeline jobs do get associated with
> the master, no jobs get build there.
>
> Gav...
>
>
> On Tue, Jan 14, 2020 at 11:37 AM Paul King <pa...@apache.org> wrote:
>
> >
> > Am I reading this correctly?
> > Nothing built on websites agent for 12 hrs:
> > https://builds.apache.org/computer/websites/builds
> >
> > On 2020/01/14 11:24:44, Zoran Regvart <zo...@regvart.com> wrote:
> > > Hi builders,
> > > could it have something to do with some builds tied to the `master`
> > node[1]?
> > >
> > > I thought it was a bad practice to tie the executors on `master`
> node...
> > >
> > > zoran
> > >
> > > [1] https://builds.apache.org/computer/(master)/builds
> > >
> > > On Tue, Jan 14, 2020 at 7:12 AM Mike Jumper <mj...@apache.org>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Is there a known issue currently affecting Jenkins? The build queue
> at
> > the
> > > > moment stands at 419 jobs and growing, yet it doesn't look like there
> > are
> > > > enough things running to account for the blockage.
> > > >
> > > > Checking the load statistics graph for Jenkins, it looks like things
> > > > suddenly took a turn at around 23:00 UTC, with queue size increasing
> > > > despite available executors going up and busy executors going down:
> > > >
> > > > https://builds.apache.org/load-statistics
> > > >
> > > > Thanks,
> > > >
> > > > - Mike
> > > >
> > >
> > >
> > > --
> > > Zoran Regvart
> > >
> >
>
>
> --
> Gav...
>
--
Zoran Regvart
Re: Jenkins queue blockage
Posted by Gavin McDonald <ip...@gmail.com>.
Hi All,
Jenkins is getting rebooted ...
Note that no jobs 'tie to master' but pipeline jobs do get associated with
the master, no jobs get build there.
Gav...
On Tue, Jan 14, 2020 at 11:37 AM Paul King <pa...@apache.org> wrote:
>
> Am I reading this correctly?
> Nothing built on websites agent for 12 hrs:
> https://builds.apache.org/computer/websites/builds
>
> On 2020/01/14 11:24:44, Zoran Regvart <zo...@regvart.com> wrote:
> > Hi builders,
> > could it have something to do with some builds tied to the `master`
> node[1]?
> >
> > I thought it was a bad practice to tie the executors on `master` node...
> >
> > zoran
> >
> > [1] https://builds.apache.org/computer/(master)/builds
> >
> > On Tue, Jan 14, 2020 at 7:12 AM Mike Jumper <mj...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > Is there a known issue currently affecting Jenkins? The build queue at
> the
> > > moment stands at 419 jobs and growing, yet it doesn't look like there
> are
> > > enough things running to account for the blockage.
> > >
> > > Checking the load statistics graph for Jenkins, it looks like things
> > > suddenly took a turn at around 23:00 UTC, with queue size increasing
> > > despite available executors going up and busy executors going down:
> > >
> > > https://builds.apache.org/load-statistics
> > >
> > > Thanks,
> > >
> > > - Mike
> > >
> >
> >
> > --
> > Zoran Regvart
> >
>
--
Gav...
Re: Jenkins queue blockage
Posted by Paul King <pa...@apache.org>.
Am I reading this correctly?
Nothing built on websites agent for 12 hrs:
https://builds.apache.org/computer/websites/builds
On 2020/01/14 11:24:44, Zoran Regvart <zo...@regvart.com> wrote:
> Hi builders,
> could it have something to do with some builds tied to the `master` node[1]?
>
> I thought it was a bad practice to tie the executors on `master` node...
>
> zoran
>
> [1] https://builds.apache.org/computer/(master)/builds
>
> On Tue, Jan 14, 2020 at 7:12 AM Mike Jumper <mj...@apache.org> wrote:
>
> > Hi all,
> >
> > Is there a known issue currently affecting Jenkins? The build queue at the
> > moment stands at 419 jobs and growing, yet it doesn't look like there are
> > enough things running to account for the blockage.
> >
> > Checking the load statistics graph for Jenkins, it looks like things
> > suddenly took a turn at around 23:00 UTC, with queue size increasing
> > despite available executors going up and busy executors going down:
> >
> > https://builds.apache.org/load-statistics
> >
> > Thanks,
> >
> > - Mike
> >
>
>
> --
> Zoran Regvart
>
Re: Jenkins queue blockage
Posted by Zoran Regvart <zo...@regvart.com>.
Hi builders,
could it have something to do with some builds tied to the `master` node[1]?
I thought it was a bad practice to tie the executors on `master` node...
zoran
[1] https://builds.apache.org/computer/(master)/builds
On Tue, Jan 14, 2020 at 7:12 AM Mike Jumper <mj...@apache.org> wrote:
> Hi all,
>
> Is there a known issue currently affecting Jenkins? The build queue at the
> moment stands at 419 jobs and growing, yet it doesn't look like there are
> enough things running to account for the blockage.
>
> Checking the load statistics graph for Jenkins, it looks like things
> suddenly took a turn at around 23:00 UTC, with queue size increasing
> despite available executors going up and busy executors going down:
>
> https://builds.apache.org/load-statistics
>
> Thanks,
>
> - Mike
>
--
Zoran Regvart