You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Gav <gm...@apache.org> on 2016/07/20 01:35:08 UTC
Puppetising Jenkins Nodes
Hi All,
We have so far added a fair few Jenkins Nodes to our Puppet3 and Git
Environment.
Those are :-
ubuntu-1 (pietas)
ubuntu-2 (penates)
ubuntu-4 (pomona)
ubuntu-5 (proserpina)
ubuntu-6 (priapus)
jenkins-ubuntu1
beam*
jenkins-power*
jenkins-test-$random (spin up on demand)
ubuntu-us1
See:-
https://github.com/apache/infrastructure-puppet/tree/deployment/data/nodes
[+ hostname.yaml]
https://github.com/apache/infrastructure-puppet/tree/deployment/modules/build_slaves
(You might find the jenkins.pp manifest informative regarding packages and
locations of)
Going forward, this should make the management of the nodes and their
packages much easier.
A real PITA previously was maintaining of the 'latest' and 'latest[1-3]'
links. We are making
good progress on improving these and will continue to do so. Please shout
up if something is a amiss.
This mail is also to inform everyone affected that we are now going to
start and puppetise
the H0-H11 nodes starting this Saturday 23rd July - one at a time. For
future maintainability and improvements we must endeavor to convert all our
ubuntu hosts - projects expect that
most if not all nodes will behave the same way and this will help achieve
that aim.
Any questions, feel free to ping the builds@ or infrastructure@ mailing
lists.
We'll start with H11 and work backwards.
Thanks!
Gav... (ASF Infra)
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
Hi again,
All H* nodes have been puppetised. With the exception of H1, H6 and H8
which are offline and no ssh, I'll do those when I get access back.
Let me know please if you spot any issues ( or god forbid, any improvements
;) )
Gav...
On Sat, Jul 23, 2016 at 9:35 AM, Gav <gm...@apache.org> wrote:
> FYI I'm going through these now, each node going to be taken down (whether
> a build is running or not) and will be down for approx 1.5 hours each
> whilst puppet populates the node with all the packages.
>
> Will let you know, when completed. One or two may have to be done when Y!
> bring the nodes back up for us as no ssh currently.
>
> Gav...
>
>
> On Fri, Jul 22, 2016 at 8:52 PM, Gav <gm...@apache.org> wrote:
>
>> Hi,
>>
>> Sorry guys, one change made was to point Maven3 Latest to a latest3 link
>> as part of the node upgrades. The H* nodes have a 'latest' link but not a
>> 'latest3' link (yet). Hence the breakage.
>>
>> I have puppetised node H2 early so do please feel free to test - I have
>> been watching and some precommit and normal builds alike seem to be
>> preforming fine.
>>
>> I'll continue on with the rest of the H* nodes in a few hours and over
>> the weekend.
>>
>> I'll be on HipChat #asfinfra (infra.chat) all w/end
>>
>> HTH
>>
>> Gav...
>>
>>
>>
>> On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
>>
>>> I have a workaround now that I believe will work across the pre-puppet
>>> and post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>>>
>>> Other interested folks, ping me if you need me.
>>>
>>> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
>>> > I am having trouble finding a maven home related env variable that
>>> works.
>>> >
>>> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
>>> > <aw...@effectivemachines.com> wrote:
>>> > >
>>> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>>> > >>
>>> > >>
>>> > >> A real PITA previously was maintaining of the 'latest' and
>>> 'latest[1-3]'
>>> > >> links. We are making
>>> > >> good progress on improving these and will continue to do so. Please
>>> shout
>>> > >> up if something is a amiss.
>>> > >
>>> > > Would this be why MAVEN_3_LATEST__HOME on some of the H
>>> nodes are empty as of a few days ago? e.g.,
>>> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
>>> > >
>>> > > Thanks.
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > busbey
>>> >
>>>
>>
>>
>
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
Hi again,
All H* nodes have been puppetised. With the exception of H1, H6 and H8
which are offline and no ssh, I'll do those when I get access back.
Let me know please if you spot any issues ( or god forbid, any improvements
;) )
Gav...
On Sat, Jul 23, 2016 at 9:35 AM, Gav <gm...@apache.org> wrote:
> FYI I'm going through these now, each node going to be taken down (whether
> a build is running or not) and will be down for approx 1.5 hours each
> whilst puppet populates the node with all the packages.
>
> Will let you know, when completed. One or two may have to be done when Y!
> bring the nodes back up for us as no ssh currently.
>
> Gav...
>
>
> On Fri, Jul 22, 2016 at 8:52 PM, Gav <gm...@apache.org> wrote:
>
>> Hi,
>>
>> Sorry guys, one change made was to point Maven3 Latest to a latest3 link
>> as part of the node upgrades. The H* nodes have a 'latest' link but not a
>> 'latest3' link (yet). Hence the breakage.
>>
>> I have puppetised node H2 early so do please feel free to test - I have
>> been watching and some precommit and normal builds alike seem to be
>> preforming fine.
>>
>> I'll continue on with the rest of the H* nodes in a few hours and over
>> the weekend.
>>
>> I'll be on HipChat #asfinfra (infra.chat) all w/end
>>
>> HTH
>>
>> Gav...
>>
>>
>>
>> On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
>>
>>> I have a workaround now that I believe will work across the pre-puppet
>>> and post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>>>
>>> Other interested folks, ping me if you need me.
>>>
>>> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
>>> > I am having trouble finding a maven home related env variable that
>>> works.
>>> >
>>> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
>>> > <aw...@effectivemachines.com> wrote:
>>> > >
>>> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>>> > >>
>>> > >>
>>> > >> A real PITA previously was maintaining of the 'latest' and
>>> 'latest[1-3]'
>>> > >> links. We are making
>>> > >> good progress on improving these and will continue to do so. Please
>>> shout
>>> > >> up if something is a amiss.
>>> > >
>>> > > Would this be why MAVEN_3_LATEST__HOME on some of the H
>>> nodes are empty as of a few days ago? e.g.,
>>> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
>>> > >
>>> > > Thanks.
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > busbey
>>> >
>>>
>>
>>
>
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
FYI I'm going through these now, each node going to be taken down (whether
a build is running or not) and will be down for approx 1.5 hours each
whilst puppet populates the node with all the packages.
Will let you know, when completed. One or two may have to be done when Y!
bring the nodes back up for us as no ssh currently.
Gav...
On Fri, Jul 22, 2016 at 8:52 PM, Gav <gm...@apache.org> wrote:
> Hi,
>
> Sorry guys, one change made was to point Maven3 Latest to a latest3 link
> as part of the node upgrades. The H* nodes have a 'latest' link but not a
> 'latest3' link (yet). Hence the breakage.
>
> I have puppetised node H2 early so do please feel free to test - I have
> been watching and some precommit and normal builds alike seem to be
> preforming fine.
>
> I'll continue on with the rest of the H* nodes in a few hours and over the
> weekend.
>
> I'll be on HipChat #asfinfra (infra.chat) all w/end
>
> HTH
>
> Gav...
>
>
>
> On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
>
>> I have a workaround now that I believe will work across the pre-puppet
>> and post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>>
>> Other interested folks, ping me if you need me.
>>
>> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
>> > I am having trouble finding a maven home related env variable that
>> works.
>> >
>> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
>> > <aw...@effectivemachines.com> wrote:
>> > >
>> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>> > >>
>> > >>
>> > >> A real PITA previously was maintaining of the 'latest' and
>> 'latest[1-3]'
>> > >> links. We are making
>> > >> good progress on improving these and will continue to do so. Please
>> shout
>> > >> up if something is a amiss.
>> > >
>> > > Would this be why MAVEN_3_LATEST__HOME on some of the H nodes
>> are empty as of a few days ago? e.g.,
>> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
>> > >
>> > > Thanks.
>> > >
>> >
>> >
>> >
>> > --
>> > busbey
>> >
>>
>
>
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
FYI I'm going through these now, each node going to be taken down (whether
a build is running or not) and will be down for approx 1.5 hours each
whilst puppet populates the node with all the packages.
Will let you know, when completed. One or two may have to be done when Y!
bring the nodes back up for us as no ssh currently.
Gav...
On Fri, Jul 22, 2016 at 8:52 PM, Gav <gm...@apache.org> wrote:
> Hi,
>
> Sorry guys, one change made was to point Maven3 Latest to a latest3 link
> as part of the node upgrades. The H* nodes have a 'latest' link but not a
> 'latest3' link (yet). Hence the breakage.
>
> I have puppetised node H2 early so do please feel free to test - I have
> been watching and some precommit and normal builds alike seem to be
> preforming fine.
>
> I'll continue on with the rest of the H* nodes in a few hours and over the
> weekend.
>
> I'll be on HipChat #asfinfra (infra.chat) all w/end
>
> HTH
>
> Gav...
>
>
>
> On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
>
>> I have a workaround now that I believe will work across the pre-puppet
>> and post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>>
>> Other interested folks, ping me if you need me.
>>
>> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
>> > I am having trouble finding a maven home related env variable that
>> works.
>> >
>> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
>> > <aw...@effectivemachines.com> wrote:
>> > >
>> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>> > >>
>> > >>
>> > >> A real PITA previously was maintaining of the 'latest' and
>> 'latest[1-3]'
>> > >> links. We are making
>> > >> good progress on improving these and will continue to do so. Please
>> shout
>> > >> up if something is a amiss.
>> > >
>> > > Would this be why MAVEN_3_LATEST__HOME on some of the H nodes
>> are empty as of a few days ago? e.g.,
>> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
>> > >
>> > > Thanks.
>> > >
>> >
>> >
>> >
>> > --
>> > busbey
>> >
>>
>
>
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
Hi,
Sorry guys, one change made was to point Maven3 Latest to a latest3 link as
part of the node upgrades. The H* nodes have a 'latest' link but not a
'latest3' link (yet). Hence the breakage.
I have puppetised node H2 early so do please feel free to test - I have
been watching and some precommit and normal builds alike seem to be
preforming fine.
I'll continue on with the rest of the H* nodes in a few hours and over the
weekend.
I'll be on HipChat #asfinfra (infra.chat) all w/end
HTH
Gav...
On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
> I have a workaround now that I believe will work across the pre-puppet and
> post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>
> Other interested folks, ping me if you need me.
>
> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
> > I am having trouble finding a maven home related env variable that works.
> >
> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
> > <aw...@effectivemachines.com> wrote:
> > >
> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
> > >>
> > >>
> > >> A real PITA previously was maintaining of the 'latest' and
> 'latest[1-3]'
> > >> links. We are making
> > >> good progress on improving these and will continue to do so. Please
> shout
> > >> up if something is a amiss.
> > >
> > > Would this be why MAVEN_3_LATEST__HOME on some of the H nodes
> are empty as of a few days ago? e.g.,
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
> > >
> > > Thanks.
> > >
> >
> >
> >
> > --
> > busbey
> >
>
Re: Puppetising Jenkins Nodes
Posted by Gav <gm...@apache.org>.
Hi,
Sorry guys, one change made was to point Maven3 Latest to a latest3 link as
part of the node upgrades. The H* nodes have a 'latest' link but not a
'latest3' link (yet). Hence the breakage.
I have puppetised node H2 early so do please feel free to test - I have
been watching and some precommit and normal builds alike seem to be
preforming fine.
I'll continue on with the rest of the H* nodes in a few hours and over the
weekend.
I'll be on HipChat #asfinfra (infra.chat) all w/end
HTH
Gav...
On Fri, Jul 22, 2016 at 12:32 PM, Sean Busbey <bu...@apache.org> wrote:
> I have a workaround now that I believe will work across the pre-puppet and
> post-puppet hosts. I'll roll it out to impacted Hadoop jobs.
>
> Other interested folks, ping me if you need me.
>
> On 2016-07-21 17:46 (-0500), Sean Busbey <bu...@cloudera.com> wrote:
> > I am having trouble finding a maven home related env variable that works.
> >
> > On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
> > <aw...@effectivemachines.com> wrote:
> > >
> > >> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
> > >>
> > >>
> > >> A real PITA previously was maintaining of the 'latest' and
> 'latest[1-3]'
> > >> links. We are making
> > >> good progress on improving these and will continue to do so. Please
> shout
> > >> up if something is a amiss.
> > >
> > > Would this be why MAVEN_3_LATEST__HOME on some of the H nodes
> are empty as of a few days ago? e.g.,
> https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
> > >
> > > Thanks.
> > >
> >
> >
> >
> > --
> > busbey
> >
>
Re: Puppetising Jenkins Nodes
Posted by Sean Busbey <bu...@cloudera.com>.
I am having trouble finding a maven home related env variable that works.
On Wed, Jul 20, 2016 at 10:37 AM, Allen Wittenauer
<aw...@effectivemachines.com> wrote:
>
>> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>>
>>
>> A real PITA previously was maintaining of the 'latest' and 'latest[1-3]'
>> links. We are making
>> good progress on improving these and will continue to do so. Please shout
>> up if something is a amiss.
>
> Would this be why MAVEN_3_LATEST__HOME on some of the H nodes are empty as of a few days ago? e.g., https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
>
> Thanks.
>
--
busbey
Re: Puppetising Jenkins Nodes
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Jul 19, 2016, at 6:35 PM, Gav <gm...@apache.org> wrote:
>
>
> A real PITA previously was maintaining of the 'latest' and 'latest[1-3]'
> links. We are making
> good progress on improving these and will continue to do so. Please shout
> up if something is a amiss.
Would this be why MAVEN_3_LATEST__HOME on some of the H nodes are empty as of a few days ago? e.g., https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/100/console
Thanks.