You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Gav <gm...@apache.org> on 2016/08/01 23:05:57 UTC
Re: Jenkins JDK Matrix - and consolidating of versions.
Hello Uwe,
Still waiting a response on what you would like doing with the Lucene VM
(lucene1-us-west) VM.
Ideally we ditch the lucene_pvm_asf module and just make it use the
build_slaves module the same
as the rest. With a note on the slave description that its still a node for
the Lucene project.
Thoughts?
Gav...
On Sat, Jul 30, 2016 at 8:33 AM, Gav <gm...@apache.org> wrote:
> Hi Uwe,
>
> On Sat, Jul 30, 2016 at 4:30 AM, Uwe Schindler <us...@apache.org>
> wrote:
>
>> Hi,
>>
>>
>>
>> after changing the setup of the Lucene Jobs, we get all jobs failing with
>> „JAVA_HOME“ not correctly defined:
>>
>>
>>
>> https://builds.apache.org/job/Lucene-Solr-Tests-master/1290/console
>>
>>
>>
>> This affects the “lucene” Jenkins node. Maybe the puppet was not
>> correctly activated there.
>>
>
> Right, I hadnt added the jenkins module to 'lucene1-us-west' - being a PMC
> owned VM I was going to ask about that.
> If you are happy for me to continue I'll get on with it today.
>
> Gav...
>
>
>
>> We need” JDK 1.7 (latest)” and “JDK 1.8 (latest)” on this machine.
>>
>
>
>
>>
>>
>> Uwe
>>
>>
>>
>> -----
>>
>> Uwe Schindler
>>
>> uschindler@apache.org
>>
>> ASF Member, Apache Lucene PMC / Committer
>>
>> Bremen, Germany
>>
>> http://lucene.apache.org/
>>
>>
>>
>> *From:* Steve Rowe [mailto:sarowe@gmail.com]
>> *Sent:* Friday, July 29, 2016 2:38 PM
>> *To:* Lucene Dev <de...@lucene.apache.org>
>> *Subject:* Fwd: Jenkins JDK Matrix - and consolidating of versions.
>>
>>
>>
>> FYI, I switched all Lucene/Solr ASF Jenkins jobs’ configuration from
>> “latest1.8” to “JDK 1.8 (latest)” (except for the 5.5 branch Java7 build,
>> which I switched from "latest1.7” to “JDK 1.7 (latest)”) - more info below.
>>
>>
>>
>> --
>>
>> Steve
>>
>> www.lucidworks.com
>>
>>
>>
>> Begin forwarded message:
>>
>>
>>
>> *From: *Gav <gm...@apache.org>
>>
>> *Subject: Jenkins JDK Matrix - and consolidating of versions.*
>>
>> *Date: *July 29, 2016 at 7:03:59 AM EDT
>>
>> *To: *builds@apache.org, "infrastructure@apache.org Infrastructure" <
>> infrastructure@apache.org>
>>
>> *Reply-To: *gmcdonald@apache.org
>>
>>
>>
>> Hi All,
>>
>> So 99% of our Jenkins build machines are now puppetised with our configs
>> in our Git repos.
>> Going forward this should make things easier to maintain the software
>> (and versions of) on the nodes.
>>
>>
>>
>> I went through all the JDK drop down options available in the Jenkins UI
>> when
>>
>> creating a build.The Confluence wiki page at :-
>>
>> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>>
>> lists all them all. When choosing a version to use I hope that this
>> information will
>> prove useful - any changes made we will keep this page upto date.
>> However, some are not currently installed on the new puppetised
>>
>> slaves (on purpose) and others we intend to consolidate down to fewer
>> versions.
>>
>> In contrast, we will be able to make newer release versions available
>> quicker.
>>
>> Once we are done with the JDKs, we'll move on to Maven, and then the rest.
>>
>> Ok to specifics,
>>
>> First major change to get out of the way is the removal of the 'auto
>> install' JDK
>>
>> versions. They overlap with installed versions and going forward we would
>> rather
>>
>> manage installation via Puppet and roll them out to all slaves in one hit.
>>
>> Looking at the table in the linked wiki page above, this means we'll be
>> removing
>>
>> eight (8) drop down options , 5 of those jdk 8, 2 x jdk 7 and 1 x jdk 6.
>> To avoid confusion we'll also be removing the auto install 'latest1.6'
>> 'latest1.7' and 'latest1.8' options.
>>
>> The best options for 'latest' versions is the ones at the top of the
>> table 'JDK 1.x (Latest)'
>> which are all installed on all nodes and managed via puppet.
>>
>> I'll point out also that we have installed 'unlimited security' versions
>> of 1.6,1.7,1.8 jdks.
>>
>> Very recent addition is also that of the JDK 9 b128 Early Access Build.
>> We hope you enjoy
>>
>> this addition and please do give feedback on this version to the builds@
>> mailing list so we can pass it on - but certainly do not plan to base your
>> releases on these 'ea' builds. I'll try my best to add new ea builds as
>> they come through.
>>
>> Last ones to mention are the IBM and Harmony versions. I'm not sure how
>> much in use
>>
>> these are so I'm holding off on installing these for now. Please do ask
>> for a specific version
>>
>> if needed. I'll drop the from the UI in time.
>>
>> I intend to drop all the auto-install versions (they are 1.6u45, 1.7u45,
>> 1.7u51, 1.8.0_05, 1.8.0_11, 1.8.0_45, 1.8.0_25, 1.8.0_66) 7 days from now,
>> along with the auto install latest links. Please plan
>>
>> on migrating your builds away from these versions before then.
>>
>> Any follow questions, please do concentrate your email responses to the
>> builds@ mailing list.
>>
>> Thanks!
>>
>> Gav... (On behalf of the Infrastructure Team)
>>
>>
>>
>
>
RE: Jenkins JDK Matrix - and consolidating of versions.
Posted by Uwe Schindler <us...@apache.org>.
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Tuesday, August 2, 2016 1:06 AM
To: Uwe Schindler <us...@apache.org>
Cc: dev@lucene.apache.org; infrastructure@apache.org Infrastructure <in...@apache.org>
Subject: Re: Jenkins JDK Matrix - and consolidating of versions.
Hello Uwe,
Still waiting a response on what you would like doing with the Lucene VM (lucene1-us-west) VM.
Ideally we ditch the lucene_pvm_asf module and just make it use the build_slaves module the same
as the rest. With a note on the slave description that its still a node for the Lucene project.
Thoughts?
Gav...
On Sat, Jul 30, 2016 at 8:33 AM, Gav <gmcdonald@apache.org <ma...@apache.org> > wrote:
Hi Uwe,
On Sat, Jul 30, 2016 at 4:30 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
after changing the setup of the Lucene Jobs, we get all jobs failing with „JAVA_HOME“ not correctly defined:
https://builds.apache.org/job/Lucene-Solr-Tests-master/1290/console
This affects the “lucene” Jenkins node. Maybe the puppet was not correctly activated there.
Right, I hadnt added the jenkins module to 'lucene1-us-west' - being a PMC owned VM I was going to ask about that.
If you are happy for me to continue I'll get on with it today.
Gav...
We need” JDK 1.7 (latest)” and “JDK 1.8 (latest)” on this machine.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
Sent: Friday, July 29, 2016 2:38 PM
To: Lucene Dev <dev@lucene.apache.org <ma...@lucene.apache.org> >
Subject: Fwd: Jenkins JDK Matrix - and consolidating of versions.
FYI, I switched all Lucene/Solr ASF Jenkins jobs’ configuration from “latest1.8” to “JDK 1.8 (latest)” (except for the 5.5 branch Java7 build, which I switched from "latest1.7” to “JDK 1.7 (latest)”) - more info below.
--
Steve
www.lucidworks.com <http://www.lucidworks.com>
Begin forwarded message:
From: Gav <gmcdonald@apache.org <ma...@apache.org> >
Subject: Jenkins JDK Matrix - and consolidating of versions.
Date: July 29, 2016 at 7:03:59 AM EDT
To: builds@apache.org <ma...@apache.org> , "infrastructure@apache.org <ma...@apache.org> Infrastructure" <infrastructure@apache.org <ma...@apache.org> >
Reply-To: gmcdonald@apache.org <ma...@apache.org>
Hi All,
So 99% of our Jenkins build machines are now puppetised with our configs in our Git repos.
Going forward this should make things easier to maintain the software (and versions of) on the nodes.
I went through all the JDK drop down options available in the Jenkins UI when
creating a build.The Confluence wiki page at :-
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
lists all them all. When choosing a version to use I hope that this information will
prove useful - any changes made we will keep this page upto date.
However, some are not currently installed on the new puppetised
slaves (on purpose) and others we intend to consolidate down to fewer versions.
In contrast, we will be able to make newer release versions available quicker.
Once we are done with the JDKs, we'll move on to Maven, and then the rest.
Ok to specifics,
First major change to get out of the way is the removal of the 'auto install' JDK
versions. They overlap with installed versions and going forward we would rather
manage installation via Puppet and roll them out to all slaves in one hit.
Looking at the table in the linked wiki page above, this means we'll be removing
eight (8) drop down options , 5 of those jdk 8, 2 x jdk 7 and 1 x jdk 6.
To avoid confusion we'll also be removing the auto install 'latest1.6' 'latest1.7' and 'latest1.8' options.
The best options for 'latest' versions is the ones at the top of the table 'JDK 1.x (Latest)'
which are all installed on all nodes and managed via puppet.
I'll point out also that we have installed 'unlimited security' versions of 1.6,1.7,1.8 jdks.
Very recent addition is also that of the JDK 9 b128 Early Access Build. We hope you enjoy
this addition and please do give feedback on this version to the builds@ mailing list so we can pass it on - but certainly do not plan to base your releases on these 'ea' builds. I'll try my best to add new ea builds as they come through.
Last ones to mention are the IBM and Harmony versions. I'm not sure how much in use
these are so I'm holding off on installing these for now. Please do ask for a specific version
if needed. I'll drop the from the UI in time.
I intend to drop all the auto-install versions (they are 1.6u45, 1.7u45, 1.7u51, 1.8.0_05, 1.8.0_11, 1.8.0_45, 1.8.0_25, 1.8.0_66) 7 days from now, along with the auto install latest links. Please plan
on migrating your builds away from these versions before then.
Any follow questions, please do concentrate your email responses to the builds@ mailing list.
Thanks!
Gav... (On behalf of the Infrastructure Team)