You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Gav <gm...@apache.org> on 2016/08/12 04:43:04 UTC
JDK, Maven, Ant versions have been consolidated.
HI All,
As per previous emails and as documented [1] [2] [3] we have now
consolidated the JDK, Maven and Ant install versions.
All versions are now installed/managed via puppet and any 'auto-install'
versions have been removed.
At the time of consolidation and removal of versions, over 200 jobs were
still configured to use deprecated versions despite the emails I've been
sending. Those 200+ jobs I have migrated myself to supported versions.
Any issues with your jobs please do create a new INFRA jira or email this
list.
Thanks,
Gav...
[1] -
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
[2] -
https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
[3] -
https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Matrix
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Svetoslav Neykov <sv...@cloudsoftcorp.com>.
Hi Gav,
Just tried "JDK 1.7 (unlimited security) 64-bit Windows only" on windows-2012-1
but I am getting the following error:
java.io.IOException <http://stacktrace.jenkins-ci.org/search?query=java.io.IOException>: Cannot run program "F:\\hudson\\tools\\java\\jdk1.7.0_79-unlimited-security/bin/java" (in directory "f:\jenkins\jenkins-slave\workspace\brooklyn-master-windows"): CreateProcess error=2, The system cannot find the file specified
from
https://builds.apache.org/job/brooklyn-master-windows/235/console
"JDK 1.7 (latest)" works fine on the same machine.
Best,
Svet.
> On 12.08.2016 г., at 7:43, Gav <gm...@apache.org> wrote:
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Matrix
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
On Thu, Aug 18, 2016 at 5:29 PM, Uwe Schindler <us...@apache.org>
wrote:
> Ah,
>
>
>
> you can delete the backup folder of old jenkins home-dir.
>
Done.
> I nuked the workspaces inside already (after sudo jenkins, otherwise we
> would have gone out of disk space), but I cannot remove the folder on its
> own.
>
>
>
> The lucene.build.properties file is now the only “customization” of this
> machine, but this one is hardware-specific just for this node.
>
>
>
ok, I'm happy for you to manage that yourselves. It is possible to maintain
it in puppet but would be more lines of code than whats in that one file ;)
Gav...
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Gav [mailto:gmcdonald@apache.org]
> *Sent:* Thursday, August 18, 2016 6:47 AM
> *To:* Uwe Schindler <us...@apache.org>
> *Cc:* dev@lucene.apache.org; builds <bu...@apache.org>
>
> *Subject:* Re: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Its fine where it is.
>
> Is there any more to do here for infra Uwe or is the Lucene Jenkins node
> all good now ?
>
> Gav...
>
>
>
> On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <us...@apache.org>
> wrote:
>
> Hi,
>
>
>
> it is not part of any distribution or in any developer checkouts. We just
> placed it there for Jenkins access to run more extensive tests – we can
> also move it to a private subversion branch, not accessible from public? We
> could then add the Jenkins user to that folder.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Sean Busbey [mailto:busbey@cloudera.com]
> *Sent:* Wednesday, August 17, 2016 1:40 PM
> *To:* builds <bu...@apache.org>
> *Cc:* gmcdonald@apache.org; dev@lucene.apache.org
>
>
> *Subject:* RE: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
> thought those licenses weren't okay to have in an ASF source repo?
>
> --
> Sean Busbey
>
> I set this up and configured Jenkins to use the test-data checked out from
> Subversion.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> > From: Steve Rowe [mailto:sarowe@gmail.com]
> > Sent: Monday, August 15, 2016 8:40 PM
> > To: dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > Thanks Uwe for moving this process along.
> >
> > +1 to make a new SVN dir at https://svn.apache.org/repos/
> asf/lucene/test-
> > data/ and put the enwiki data file there.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >
> > > Hi,
> > >
> > > I thought about the wikipedia test files used by Jenkins: I think I
> would
> > commit them to SVN inside the Lucene/Solr project’s folder. The
> Nightly-Test
> > Jenkins jobs that use them could simply check them out into a separate
> dir of
> > the workspace. Byt that it is easier for us to update them. If you also
> think
> > this is fine, we can leave it like that.
> > >
> > > The ~jenkins/lucene.build.properties file is (as said before)
> > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> > create a puppet module out of it, but this would prevent us from
> optimizing
> > or changing them quickly per project requirements.
> > >
> > > Finally the “ant ivy-bootstrap” task was for now separated into a
> Jenkins
> > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> > dependency to the “ant jenkins” and other targets which are solely
> triggered
> > by Jenkins, so separately bootstrapping is no longer required (please
> keep in
> > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> > workspace – this is why it’s done separately!). I just have to fix the
> Jenkins-
> > specific targets in our build.xml in Git repo to only do the actual
> > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
> machine.
> > Then it’s a no-op on most builds. For now the separate Jenkins job is
> enough
> > to quickly do the bootstrap as workaround.
> > >
> > > If nobody stops me from committing the following file into SVN:
> > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > > total 2.9G
> > > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> > >
> > > I will do this later this evening and change the Jenkins builds to do
> an extra
> > checkout into the workspace next to Git.
> > > I’d suggest to place the folder contents of ~/lucene-data here:
> > https://svn.apache.org/repos/asf/lucene/test-data
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Uwe Schindler [mailto:uschindler@apache.org]
> > > Sent: Monday, August 15, 2016 12:47 PM
> > > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> > <us...@apache.org>
> > > Cc: 'builds' <bu...@apache.org>
> > > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> > >
> > > Hi,
> > >
> > > the test files could be a puppet module.
> > >
> > > The lucene.build.properties in the Jenkins home dir is a “hardware
> specific”
> > file, as it allows to configure the number of parallel runners and where
> the
> > special test files are – one would not be able to use it on any other
> jenkins
> > slave. If it is missing, defaults are used, which are not good for
> optimal CPU
> > use. We could add them also to the puppet module, but this would make it
> > harder for use to change the file easily.
> > >
> > > All other files are unmodified.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Gav [mailto:gmcdonald@apache.org]
> > > Sent: Monday, August 15, 2016 5:36 AM
> > > To: Uwe Schindler <us...@apache.org>
> > > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >
> > > I'll make a copy of the jenkins home dir and get started.
> > > It might be an idea to add those test files to puppet wdyt?
> > > (I see nothing private about them)
> > >
> > > Gav...
> > >
> > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >> Hi Gav,
> > >>
> > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this
> spammed
> > the mailing list with like 50 mails today. Can you send me a msg when you
> > installed the required puppet modules?
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> uschindler@apache.org
> > >> ASF Member, Apache Lucene PMC / Committer
> > >> Bremen, Germany
> > >> http://lucene.apache.org/
> > >>
> > >> From: Gav [mailto:gmcdonald@apache.org]
> > >> Sent: Friday, August 12, 2016 11:18 AM
> > >> To: Uwe Schindler <us...@apache.org>
> > >> Cc: builds <bu...@apache.org>
> > >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >>
> > >> Hi Uwe,
> > >>
> > >> Will get on with your vm tomorrow.
> > >>
> > >> Gav...
> > >>
> > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org
> >
> > wrote:
> > >>> Hi,
> > >>>
> > >>> on the lucene-jenkins machine, the packages are still missing. I
> think you
> > have to install the jenkins puppet module. Here the message:
> > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> > 1.8.4"
> > >>>
> > >>> See my e-mail from last week about using the "official" jenkins
> puppet
> > module (answer to your question). Unless our customizations in the home
> > dir keep preserved, we are fine with that:
> > >>>
> > >>> My mail:
> > >>>
> > >>> ====================================
> > >>> Hi Gav,
> > >>>
> > >>> sorry for responding late – I am currently on vacation!
> > >>>
> > >>> About your questions: In the past we required some extra modules
> (like
> > older Subversion to be compatible with the older Jenkins Checkouts). As
> we
> > are on Git now and the build scripts use JGit to fetch revision numbers
> and
> > check for clean checkouts, this is obsolete. So if you want to install
> the
> > “Jenkins puppet module”, we are fine.
> > >>>
> > >>> There is one special case for Lucene:
> > >>> We have a hardware-specific “lucene.build.properties” file in
> Jenkin’s
> > root and also some additional “test files” (large wikiepdia checkouts)
> that are
> > used for nightly tests. If the puppet module does not delete those files
> in
> > current ~/jenkins, it is ok. We would still need access to the machine
> and
> > sudo to jenkins.
> > >>>
> > >>> If this is all fine, go ahead!
> > >>>
> > >>> Uwe
> > >>> =======================================
> > >>>
> > >>> Uwe
> > >>>
> > >>> -----
> > >>> Uwe Schindler
> > >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >>> http://www.thetaphi.de
> > >>> eMail: uwe@thetaphi.de
> > >>>
> > >>> > -----Original Message-----
> > >>> > From: Gav [mailto:gmcdonald@apache.org]
> > >>> > Sent: Friday, August 12, 2016 6:43 AM
> > >>> > To: builds <bu...@apache.org>
> > >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> > >>> >
> > >>> > HI All,
> > >>> >
> > >>> > As per previous emails and as documented [1] [2] [3] we have now
> > >>> > consolidated the JDK, Maven and Ant install versions.
> > >>> >
> > >>> > All versions are now installed/managed via puppet and any 'auto-
> > install'
> > >>> > versions have been removed.
> > >>> >
> > >>> > At the time of consolidation and removal of versions, over 200 jobs
> > were
> > >>> > still configured to use deprecated versions despite the emails
> I've been
> > >>> > sending. Those 200+ jobs I have migrated myself to supported
> > versions.
> > >>> >
> > >>> > Any issues with your jobs please do create a new INFRA jira or
> email
> > this
> > >>> > list.
> > >>> >
> > >>> > Thanks,
> > >>> >
> > >>> > Gav...
> > >>> >
> > >>> > [1] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > >>> > [2] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > >>> > [3] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > >>> > rix
> > >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
On Thu, Aug 18, 2016 at 5:29 PM, Uwe Schindler <us...@apache.org>
wrote:
> Ah,
>
>
>
> you can delete the backup folder of old jenkins home-dir.
>
Done.
> I nuked the workspaces inside already (after sudo jenkins, otherwise we
> would have gone out of disk space), but I cannot remove the folder on its
> own.
>
>
>
> The lucene.build.properties file is now the only “customization” of this
> machine, but this one is hardware-specific just for this node.
>
>
>
ok, I'm happy for you to manage that yourselves. It is possible to maintain
it in puppet but would be more lines of code than whats in that one file ;)
Gav...
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Gav [mailto:gmcdonald@apache.org]
> *Sent:* Thursday, August 18, 2016 6:47 AM
> *To:* Uwe Schindler <us...@apache.org>
> *Cc:* dev@lucene.apache.org; builds <bu...@apache.org>
>
> *Subject:* Re: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Its fine where it is.
>
> Is there any more to do here for infra Uwe or is the Lucene Jenkins node
> all good now ?
>
> Gav...
>
>
>
> On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <us...@apache.org>
> wrote:
>
> Hi,
>
>
>
> it is not part of any distribution or in any developer checkouts. We just
> placed it there for Jenkins access to run more extensive tests – we can
> also move it to a private subversion branch, not accessible from public? We
> could then add the Jenkins user to that folder.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Sean Busbey [mailto:busbey@cloudera.com]
> *Sent:* Wednesday, August 17, 2016 1:40 PM
> *To:* builds <bu...@apache.org>
> *Cc:* gmcdonald@apache.org; dev@lucene.apache.org
>
>
> *Subject:* RE: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
> thought those licenses weren't okay to have in an ASF source repo?
>
> --
> Sean Busbey
>
> I set this up and configured Jenkins to use the test-data checked out from
> Subversion.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> > From: Steve Rowe [mailto:sarowe@gmail.com]
> > Sent: Monday, August 15, 2016 8:40 PM
> > To: dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > Thanks Uwe for moving this process along.
> >
> > +1 to make a new SVN dir at https://svn.apache.org/repos/
> asf/lucene/test-
> > data/ and put the enwiki data file there.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >
> > > Hi,
> > >
> > > I thought about the wikipedia test files used by Jenkins: I think I
> would
> > commit them to SVN inside the Lucene/Solr project’s folder. The
> Nightly-Test
> > Jenkins jobs that use them could simply check them out into a separate
> dir of
> > the workspace. Byt that it is easier for us to update them. If you also
> think
> > this is fine, we can leave it like that.
> > >
> > > The ~jenkins/lucene.build.properties file is (as said before)
> > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> > create a puppet module out of it, but this would prevent us from
> optimizing
> > or changing them quickly per project requirements.
> > >
> > > Finally the “ant ivy-bootstrap” task was for now separated into a
> Jenkins
> > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> > dependency to the “ant jenkins” and other targets which are solely
> triggered
> > by Jenkins, so separately bootstrapping is no longer required (please
> keep in
> > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> > workspace – this is why it’s done separately!). I just have to fix the
> Jenkins-
> > specific targets in our build.xml in Git repo to only do the actual
> > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
> machine.
> > Then it’s a no-op on most builds. For now the separate Jenkins job is
> enough
> > to quickly do the bootstrap as workaround.
> > >
> > > If nobody stops me from committing the following file into SVN:
> > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > > total 2.9G
> > > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> > >
> > > I will do this later this evening and change the Jenkins builds to do
> an extra
> > checkout into the workspace next to Git.
> > > I’d suggest to place the folder contents of ~/lucene-data here:
> > https://svn.apache.org/repos/asf/lucene/test-data
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Uwe Schindler [mailto:uschindler@apache.org]
> > > Sent: Monday, August 15, 2016 12:47 PM
> > > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> > <us...@apache.org>
> > > Cc: 'builds' <bu...@apache.org>
> > > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> > >
> > > Hi,
> > >
> > > the test files could be a puppet module.
> > >
> > > The lucene.build.properties in the Jenkins home dir is a “hardware
> specific”
> > file, as it allows to configure the number of parallel runners and where
> the
> > special test files are – one would not be able to use it on any other
> jenkins
> > slave. If it is missing, defaults are used, which are not good for
> optimal CPU
> > use. We could add them also to the puppet module, but this would make it
> > harder for use to change the file easily.
> > >
> > > All other files are unmodified.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Gav [mailto:gmcdonald@apache.org]
> > > Sent: Monday, August 15, 2016 5:36 AM
> > > To: Uwe Schindler <us...@apache.org>
> > > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >
> > > I'll make a copy of the jenkins home dir and get started.
> > > It might be an idea to add those test files to puppet wdyt?
> > > (I see nothing private about them)
> > >
> > > Gav...
> > >
> > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >> Hi Gav,
> > >>
> > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this
> spammed
> > the mailing list with like 50 mails today. Can you send me a msg when you
> > installed the required puppet modules?
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> uschindler@apache.org
> > >> ASF Member, Apache Lucene PMC / Committer
> > >> Bremen, Germany
> > >> http://lucene.apache.org/
> > >>
> > >> From: Gav [mailto:gmcdonald@apache.org]
> > >> Sent: Friday, August 12, 2016 11:18 AM
> > >> To: Uwe Schindler <us...@apache.org>
> > >> Cc: builds <bu...@apache.org>
> > >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >>
> > >> Hi Uwe,
> > >>
> > >> Will get on with your vm tomorrow.
> > >>
> > >> Gav...
> > >>
> > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org
> >
> > wrote:
> > >>> Hi,
> > >>>
> > >>> on the lucene-jenkins machine, the packages are still missing. I
> think you
> > have to install the jenkins puppet module. Here the message:
> > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> > 1.8.4"
> > >>>
> > >>> See my e-mail from last week about using the "official" jenkins
> puppet
> > module (answer to your question). Unless our customizations in the home
> > dir keep preserved, we are fine with that:
> > >>>
> > >>> My mail:
> > >>>
> > >>> ====================================
> > >>> Hi Gav,
> > >>>
> > >>> sorry for responding late – I am currently on vacation!
> > >>>
> > >>> About your questions: In the past we required some extra modules
> (like
> > older Subversion to be compatible with the older Jenkins Checkouts). As
> we
> > are on Git now and the build scripts use JGit to fetch revision numbers
> and
> > check for clean checkouts, this is obsolete. So if you want to install
> the
> > “Jenkins puppet module”, we are fine.
> > >>>
> > >>> There is one special case for Lucene:
> > >>> We have a hardware-specific “lucene.build.properties” file in
> Jenkin’s
> > root and also some additional “test files” (large wikiepdia checkouts)
> that are
> > used for nightly tests. If the puppet module does not delete those files
> in
> > current ~/jenkins, it is ok. We would still need access to the machine
> and
> > sudo to jenkins.
> > >>>
> > >>> If this is all fine, go ahead!
> > >>>
> > >>> Uwe
> > >>> =======================================
> > >>>
> > >>> Uwe
> > >>>
> > >>> -----
> > >>> Uwe Schindler
> > >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >>> http://www.thetaphi.de
> > >>> eMail: uwe@thetaphi.de
> > >>>
> > >>> > -----Original Message-----
> > >>> > From: Gav [mailto:gmcdonald@apache.org]
> > >>> > Sent: Friday, August 12, 2016 6:43 AM
> > >>> > To: builds <bu...@apache.org>
> > >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> > >>> >
> > >>> > HI All,
> > >>> >
> > >>> > As per previous emails and as documented [1] [2] [3] we have now
> > >>> > consolidated the JDK, Maven and Ant install versions.
> > >>> >
> > >>> > All versions are now installed/managed via puppet and any 'auto-
> > install'
> > >>> > versions have been removed.
> > >>> >
> > >>> > At the time of consolidation and removal of versions, over 200 jobs
> > were
> > >>> > still configured to use deprecated versions despite the emails
> I've been
> > >>> > sending. Those 200+ jobs I have migrated myself to supported
> > versions.
> > >>> >
> > >>> > Any issues with your jobs please do create a new INFRA jira or
> email
> > this
> > >>> > list.
> > >>> >
> > >>> > Thanks,
> > >>> >
> > >>> > Gav...
> > >>> >
> > >>> > [1] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > >>> > [2] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > >>> > [3] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > >>> > rix
> > >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Ah,
you can delete the backup folder of old jenkins home-dir. I nuked the workspaces inside already (after sudo jenkins, otherwise we would have gone out of disk space), but I cannot remove the folder on its own.
The lucene.build.properties file is now the only “customization” of this machine, but this one is hardware-specific just for this node.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Thursday, August 18, 2016 6:47 AM
To: Uwe Schindler <us...@apache.org>
Cc: dev@lucene.apache.org; builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com <ma...@cloudera.com> ]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <builds@apache.org <ma...@apache.org> >
Cc: gmcdonald@apache.org <ma...@apache.org> ; dev@lucene.apache.org <ma...@lucene.apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org <ma...@apache.org> ; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
looks all good now. It’s working since Monday morning without any problems.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Thursday, August 18, 2016 6:47 AM
To: Uwe Schindler <us...@apache.org>
Cc: dev@lucene.apache.org; builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com <ma...@cloudera.com> ]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <builds@apache.org <ma...@apache.org> >
Cc: gmcdonald@apache.org <ma...@apache.org> ; dev@lucene.apache.org <ma...@lucene.apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
looks all good now. It’s working since Monday morning without any problems.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Thursday, August 18, 2016 6:47 AM
To: Uwe Schindler <us...@apache.org>
Cc: dev@lucene.apache.org; builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com <ma...@cloudera.com> ]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <builds@apache.org <ma...@apache.org> >
Cc: gmcdonald@apache.org <ma...@apache.org> ; dev@lucene.apache.org <ma...@lucene.apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Ah,
you can delete the backup folder of old jenkins home-dir. I nuked the workspaces inside already (after sudo jenkins, otherwise we would have gone out of disk space), but I cannot remove the folder on its own.
The lucene.build.properties file is now the only “customization” of this machine, but this one is hardware-specific just for this node.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Thursday, August 18, 2016 6:47 AM
To: Uwe Schindler <us...@apache.org>
Cc: dev@lucene.apache.org; builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com <ma...@cloudera.com> ]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <builds@apache.org <ma...@apache.org> >
Cc: gmcdonald@apache.org <ma...@apache.org> ; dev@lucene.apache.org <ma...@lucene.apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org <ma...@apache.org> ; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node
all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <us...@apache.org>
wrote:
> Hi,
>
>
>
> it is not part of any distribution or in any developer checkouts. We just
> placed it there for Jenkins access to run more extensive tests – we can
> also move it to a private subversion branch, not accessible from public? We
> could then add the Jenkins user to that folder.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Sean Busbey [mailto:busbey@cloudera.com]
> *Sent:* Wednesday, August 17, 2016 1:40 PM
> *To:* builds <bu...@apache.org>
> *Cc:* gmcdonald@apache.org; dev@lucene.apache.org
>
> *Subject:* RE: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
> thought those licenses weren't okay to have in an ASF source repo?
>
> --
> Sean Busbey
>
> I set this up and configured Jenkins to use the test-data checked out from
> Subversion.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> > From: Steve Rowe [mailto:sarowe@gmail.com]
> > Sent: Monday, August 15, 2016 8:40 PM
> > To: dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > Thanks Uwe for moving this process along.
> >
> > +1 to make a new SVN dir at https://svn.apache.org/repos/
> asf/lucene/test-
> > data/ and put the enwiki data file there.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >
> > > Hi,
> > >
> > > I thought about the wikipedia test files used by Jenkins: I think I
> would
> > commit them to SVN inside the Lucene/Solr project’s folder. The
> Nightly-Test
> > Jenkins jobs that use them could simply check them out into a separate
> dir of
> > the workspace. Byt that it is easier for us to update them. If you also
> think
> > this is fine, we can leave it like that.
> > >
> > > The ~jenkins/lucene.build.properties file is (as said before)
> > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> > create a puppet module out of it, but this would prevent us from
> optimizing
> > or changing them quickly per project requirements.
> > >
> > > Finally the “ant ivy-bootstrap” task was for now separated into a
> Jenkins
> > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> > dependency to the “ant jenkins” and other targets which are solely
> triggered
> > by Jenkins, so separately bootstrapping is no longer required (please
> keep in
> > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> > workspace – this is why it’s done separately!). I just have to fix the
> Jenkins-
> > specific targets in our build.xml in Git repo to only do the actual
> > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
> machine.
> > Then it’s a no-op on most builds. For now the separate Jenkins job is
> enough
> > to quickly do the bootstrap as workaround.
> > >
> > > If nobody stops me from committing the following file into SVN:
> > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > > total 2.9G
> > > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> > >
> > > I will do this later this evening and change the Jenkins builds to do
> an extra
> > checkout into the workspace next to Git.
> > > I’d suggest to place the folder contents of ~/lucene-data here:
> > https://svn.apache.org/repos/asf/lucene/test-data
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Uwe Schindler [mailto:uschindler@apache.org]
> > > Sent: Monday, August 15, 2016 12:47 PM
> > > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> > <us...@apache.org>
> > > Cc: 'builds' <bu...@apache.org>
> > > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> > >
> > > Hi,
> > >
> > > the test files could be a puppet module.
> > >
> > > The lucene.build.properties in the Jenkins home dir is a “hardware
> specific”
> > file, as it allows to configure the number of parallel runners and where
> the
> > special test files are – one would not be able to use it on any other
> jenkins
> > slave. If it is missing, defaults are used, which are not good for
> optimal CPU
> > use. We could add them also to the puppet module, but this would make it
> > harder for use to change the file easily.
> > >
> > > All other files are unmodified.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Gav [mailto:gmcdonald@apache.org]
> > > Sent: Monday, August 15, 2016 5:36 AM
> > > To: Uwe Schindler <us...@apache.org>
> > > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >
> > > I'll make a copy of the jenkins home dir and get started.
> > > It might be an idea to add those test files to puppet wdyt?
> > > (I see nothing private about them)
> > >
> > > Gav...
> > >
> > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >> Hi Gav,
> > >>
> > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this
> spammed
> > the mailing list with like 50 mails today. Can you send me a msg when you
> > installed the required puppet modules?
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> uschindler@apache.org
> > >> ASF Member, Apache Lucene PMC / Committer
> > >> Bremen, Germany
> > >> http://lucene.apache.org/
> > >>
> > >> From: Gav [mailto:gmcdonald@apache.org]
> > >> Sent: Friday, August 12, 2016 11:18 AM
> > >> To: Uwe Schindler <us...@apache.org>
> > >> Cc: builds <bu...@apache.org>
> > >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >>
> > >> Hi Uwe,
> > >>
> > >> Will get on with your vm tomorrow.
> > >>
> > >> Gav...
> > >>
> > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org
> >
> > wrote:
> > >>> Hi,
> > >>>
> > >>> on the lucene-jenkins machine, the packages are still missing. I
> think you
> > have to install the jenkins puppet module. Here the message:
> > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> > 1.8.4"
> > >>>
> > >>> See my e-mail from last week about using the "official" jenkins
> puppet
> > module (answer to your question). Unless our customizations in the home
> > dir keep preserved, we are fine with that:
> > >>>
> > >>> My mail:
> > >>>
> > >>> ====================================
> > >>> Hi Gav,
> > >>>
> > >>> sorry for responding late – I am currently on vacation!
> > >>>
> > >>> About your questions: In the past we required some extra modules
> (like
> > older Subversion to be compatible with the older Jenkins Checkouts). As
> we
> > are on Git now and the build scripts use JGit to fetch revision numbers
> and
> > check for clean checkouts, this is obsolete. So if you want to install
> the
> > “Jenkins puppet module”, we are fine.
> > >>>
> > >>> There is one special case for Lucene:
> > >>> We have a hardware-specific “lucene.build.properties” file in
> Jenkin’s
> > root and also some additional “test files” (large wikiepdia checkouts)
> that are
> > used for nightly tests. If the puppet module does not delete those files
> in
> > current ~/jenkins, it is ok. We would still need access to the machine
> and
> > sudo to jenkins.
> > >>>
> > >>> If this is all fine, go ahead!
> > >>>
> > >>> Uwe
> > >>> =======================================
> > >>>
> > >>> Uwe
> > >>>
> > >>> -----
> > >>> Uwe Schindler
> > >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >>> http://www.thetaphi.de
> > >>> eMail: uwe@thetaphi.de
> > >>>
> > >>> > -----Original Message-----
> > >>> > From: Gav [mailto:gmcdonald@apache.org]
> > >>> > Sent: Friday, August 12, 2016 6:43 AM
> > >>> > To: builds <bu...@apache.org>
> > >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> > >>> >
> > >>> > HI All,
> > >>> >
> > >>> > As per previous emails and as documented [1] [2] [3] we have now
> > >>> > consolidated the JDK, Maven and Ant install versions.
> > >>> >
> > >>> > All versions are now installed/managed via puppet and any 'auto-
> > install'
> > >>> > versions have been removed.
> > >>> >
> > >>> > At the time of consolidation and removal of versions, over 200 jobs
> > were
> > >>> > still configured to use deprecated versions despite the emails
> I've been
> > >>> > sending. Those 200+ jobs I have migrated myself to supported
> > versions.
> > >>> >
> > >>> > Any issues with your jobs please do create a new INFRA jira or
> email
> > this
> > >>> > list.
> > >>> >
> > >>> > Thanks,
> > >>> >
> > >>> > Gav...
> > >>> >
> > >>> > [1] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > >>> > [2] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > >>> > [3] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > >>> > rix
> > >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
Its fine where it is.
Is there any more to do here for infra Uwe or is the Lucene Jenkins node
all good now ?
Gav...
On Wed, Aug 17, 2016 at 10:33 PM, Uwe Schindler <us...@apache.org>
wrote:
> Hi,
>
>
>
> it is not part of any distribution or in any developer checkouts. We just
> placed it there for Jenkins access to run more extensive tests – we can
> also move it to a private subversion branch, not accessible from public? We
> could then add the Jenkins user to that folder.
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Sean Busbey [mailto:busbey@cloudera.com]
> *Sent:* Wednesday, August 17, 2016 1:40 PM
> *To:* builds <bu...@apache.org>
> *Cc:* gmcdonald@apache.org; dev@lucene.apache.org
>
> *Subject:* RE: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
> thought those licenses weren't okay to have in an ASF source repo?
>
> --
> Sean Busbey
>
> I set this up and configured Jenkins to use the test-data checked out from
> Subversion.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> > From: Steve Rowe [mailto:sarowe@gmail.com]
> > Sent: Monday, August 15, 2016 8:40 PM
> > To: dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > Thanks Uwe for moving this process along.
> >
> > +1 to make a new SVN dir at https://svn.apache.org/repos/
> asf/lucene/test-
> > data/ and put the enwiki data file there.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> > > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >
> > > Hi,
> > >
> > > I thought about the wikipedia test files used by Jenkins: I think I
> would
> > commit them to SVN inside the Lucene/Solr project’s folder. The
> Nightly-Test
> > Jenkins jobs that use them could simply check them out into a separate
> dir of
> > the workspace. Byt that it is easier for us to update them. If you also
> think
> > this is fine, we can leave it like that.
> > >
> > > The ~jenkins/lucene.build.properties file is (as said before)
> > machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> > create a puppet module out of it, but this would prevent us from
> optimizing
> > or changing them quickly per project requirements.
> > >
> > > Finally the “ant ivy-bootstrap” task was for now separated into a
> Jenkins
> > job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> > Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> > dependency to the “ant jenkins” and other targets which are solely
> triggered
> > by Jenkins, so separately bootstrapping is no longer required (please
> keep in
> > mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> > workspace – this is why it’s done separately!). I just have to fix the
> Jenkins-
> > specific targets in our build.xml in Git repo to only do the actual
> > bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
> machine.
> > Then it’s a no-op on most builds. For now the separate Jenkins job is
> enough
> > to quickly do the bootstrap as workaround.
> > >
> > > If nobody stops me from committing the following file into SVN:
> > > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > > total 2.9G
> > > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> > >
> > > I will do this later this evening and change the Jenkins builds to do
> an extra
> > checkout into the workspace next to Git.
> > > I’d suggest to place the folder contents of ~/lucene-data here:
> > https://svn.apache.org/repos/asf/lucene/test-data
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Uwe Schindler [mailto:uschindler@apache.org]
> > > Sent: Monday, August 15, 2016 12:47 PM
> > > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> > <us...@apache.org>
> > > Cc: 'builds' <bu...@apache.org>
> > > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> > >
> > > Hi,
> > >
> > > the test files could be a puppet module.
> > >
> > > The lucene.build.properties in the Jenkins home dir is a “hardware
> specific”
> > file, as it allows to configure the number of parallel runners and where
> the
> > special test files are – one would not be able to use it on any other
> jenkins
> > slave. If it is missing, defaults are used, which are not good for
> optimal CPU
> > use. We could add them also to the puppet module, but this would make it
> > harder for use to change the file easily.
> > >
> > > All other files are unmodified.
> > >
> > > Uwe
> > >
> > > -----
> > > Uwe Schindler
> > > uschindler@apache.org
> > > ASF Member, Apache Lucene PMC / Committer
> > > Bremen, Germany
> > > http://lucene.apache.org/
> > >
> > > From: Gav [mailto:gmcdonald@apache.org]
> > > Sent: Monday, August 15, 2016 5:36 AM
> > > To: Uwe Schindler <us...@apache.org>
> > > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >
> > > I'll make a copy of the jenkins home dir and get started.
> > > It might be an idea to add those test files to puppet wdyt?
> > > (I see nothing private about them)
> > >
> > > Gav...
> > >
> > > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> > wrote:
> > >> Hi Gav,
> > >>
> > >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this
> spammed
> > the mailing list with like 50 mails today. Can you send me a msg when you
> > installed the required puppet modules?
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> uschindler@apache.org
> > >> ASF Member, Apache Lucene PMC / Committer
> > >> Bremen, Germany
> > >> http://lucene.apache.org/
> > >>
> > >> From: Gav [mailto:gmcdonald@apache.org]
> > >> Sent: Friday, August 12, 2016 11:18 AM
> > >> To: Uwe Schindler <us...@apache.org>
> > >> Cc: builds <bu...@apache.org>
> > >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> > >>
> > >> Hi Uwe,
> > >>
> > >> Will get on with your vm tomorrow.
> > >>
> > >> Gav...
> > >>
> > >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org
> >
> > wrote:
> > >>> Hi,
> > >>>
> > >>> on the lucene-jenkins machine, the packages are still missing. I
> think you
> > have to install the jenkins puppet module. Here the message:
> > >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> > 1.8.4"
> > >>>
> > >>> See my e-mail from last week about using the "official" jenkins
> puppet
> > module (answer to your question). Unless our customizations in the home
> > dir keep preserved, we are fine with that:
> > >>>
> > >>> My mail:
> > >>>
> > >>> ====================================
> > >>> Hi Gav,
> > >>>
> > >>> sorry for responding late – I am currently on vacation!
> > >>>
> > >>> About your questions: In the past we required some extra modules
> (like
> > older Subversion to be compatible with the older Jenkins Checkouts). As
> we
> > are on Git now and the build scripts use JGit to fetch revision numbers
> and
> > check for clean checkouts, this is obsolete. So if you want to install
> the
> > “Jenkins puppet module”, we are fine.
> > >>>
> > >>> There is one special case for Lucene:
> > >>> We have a hardware-specific “lucene.build.properties” file in
> Jenkin’s
> > root and also some additional “test files” (large wikiepdia checkouts)
> that are
> > used for nightly tests. If the puppet module does not delete those files
> in
> > current ~/jenkins, it is ok. We would still need access to the machine
> and
> > sudo to jenkins.
> > >>>
> > >>> If this is all fine, go ahead!
> > >>>
> > >>> Uwe
> > >>> =======================================
> > >>>
> > >>> Uwe
> > >>>
> > >>> -----
> > >>> Uwe Schindler
> > >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> > >>> http://www.thetaphi.de
> > >>> eMail: uwe@thetaphi.de
> > >>>
> > >>> > -----Original Message-----
> > >>> > From: Gav [mailto:gmcdonald@apache.org]
> > >>> > Sent: Friday, August 12, 2016 6:43 AM
> > >>> > To: builds <bu...@apache.org>
> > >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> > >>> >
> > >>> > HI All,
> > >>> >
> > >>> > As per previous emails and as documented [1] [2] [3] we have now
> > >>> > consolidated the JDK, Maven and Ant install versions.
> > >>> >
> > >>> > All versions are now installed/managed via puppet and any 'auto-
> > install'
> > >>> > versions have been removed.
> > >>> >
> > >>> > At the time of consolidation and removal of versions, over 200 jobs
> > were
> > >>> > still configured to use deprecated versions despite the emails
> I've been
> > >>> > sending. Those 200+ jobs I have migrated myself to supported
> > versions.
> > >>> >
> > >>> > Any issues with your jobs please do create a new INFRA jira or
> email
> > this
> > >>> > list.
> > >>> >
> > >>> > Thanks,
> > >>> >
> > >>> > Gav...
> > >>> >
> > >>> > [1] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > >>> > [2] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > >>> > [3] -
> > >>> >
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > >>> > rix
> > >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <bu...@apache.org>
Cc: gmcdonald@apache.org; dev@lucene.apache.org
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org <ma...@apache.org> ; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
it is not part of any distribution or in any developer checkouts. We just placed it there for Jenkins access to run more extensive tests – we can also move it to a private subversion branch, not accessible from public? We could then add the Jenkins user to that folder.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, August 17, 2016 1:40 PM
To: builds <bu...@apache.org>
Cc: gmcdonald@apache.org; dev@lucene.apache.org
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com <ma...@gmail.com> ]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org <ma...@lucene.apache.org>
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com <http://www.lucidworks.com>
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org <ma...@lucene.apache.org> ; gmcdonald@apache.org <ma...@apache.org> ; 'Uwe Schindler'
> <uschindler@apache.org <ma...@apache.org> >
> > Cc: 'builds' <builds@apache.org <ma...@apache.org> >
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org <ma...@apache.org>
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> > Cc: builds <builds@apache.org <ma...@apache.org> >; dev@lucene.apache.org <ma...@lucene.apache.org>
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org <ma...@apache.org>
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> >> Cc: builds <builds@apache.org <ma...@apache.org> >
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <builds@apache.org <ma...@apache.org> >
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <ma...@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <ma...@lucene.apache.org>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Sean Busbey <bu...@cloudera.com>.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from
Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I
would
> commit them to SVN inside the Lucene/Solr project’s folder. The
Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate
dir of
> the workspace. Byt that it is easier for us to update them. If you also
think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from
optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a
Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely
triggered
> by Jenkins, so separately bootstrapping is no longer required (please
keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the
Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is
enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do
an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> <us...@apache.org>
> > Cc: 'builds' <bu...@apache.org>
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware
specific”
> file, as it allows to configure the number of parallel runners and where
the
> special test files are – one would not be able to use it on any other
jenkins
> slave. If it is missing, defaults are used, which are not good for
optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <us...@apache.org>
> > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <us...@apache.org>
> >> Cc: builds <bu...@apache.org>
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I
think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers
and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts)
that are
> used for nightly tests. If the puppet module does not delete those files
in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <bu...@apache.org>
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've
been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Sean Busbey <bu...@cloudera.com>.
Isn't the wikipedia english corpus licensed either CC-BY-SA or GFDL? I
thought those licenses weren't okay to have in an ASF source repo?
--
Sean Busbey
I set this up and configured Jenkins to use the test-data checked out from
Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I
would
> commit them to SVN inside the Lucene/Solr project’s folder. The
Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate
dir of
> the workspace. Byt that it is easier for us to update them. If you also
think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from
optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a
Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely
triggered
> by Jenkins, so separately bootstrapping is no longer required (please
keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the
Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’
machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is
enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do
an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> <us...@apache.org>
> > Cc: 'builds' <bu...@apache.org>
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware
specific”
> file, as it allows to configure the number of parallel runners and where
the
> special test files are – one would not be able to use it on any other
jenkins
> slave. If it is missing, defaults are used, which are not good for
optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <us...@apache.org>
> > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <us...@apache.org>
> >> Cc: builds <bu...@apache.org>
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I
think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers
and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts)
that are
> used for nightly tests. If the puppet module does not delete those files
in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <bu...@apache.org>
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've
been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> <us...@apache.org>
> > Cc: 'builds' <bu...@apache.org>
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <us...@apache.org>
> > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <us...@apache.org>
> >> Cc: builds <bu...@apache.org>
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <bu...@apache.org>
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
I set this up and configured Jenkins to use the test-data checked out from Subversion.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
> From: Steve Rowe [mailto:sarowe@gmail.com]
> Sent: Monday, August 15, 2016 8:40 PM
> To: dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> Thanks Uwe for moving this process along.
>
> +1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-
> data/ and put the enwiki data file there.
>
> --
> Steve
> www.lucidworks.com
>
> > On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >
> > Hi,
> >
> > I thought about the wikipedia test files used by Jenkins: I think I would
> commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test
> Jenkins jobs that use them could simply check them out into a separate dir of
> the workspace. Byt that it is easier for us to update them. If you also think
> this is fine, we can leave it like that.
> >
> > The ~jenkins/lucene.build.properties file is (as said before)
> machine/hardware specific. I’d keep them in Jenkins’ home dir. We may
> create a puppet module out of it, but this would prevent us from optimizing
> or changing them quickly per project requirements.
> >
> > Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins
> job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-
> Bootstrap/). In the future, we can add the “ivy-bootstrap” target as
> dependency to the “ant jenkins” and other targets which are solely triggered
> by Jenkins, so separately bootstrapping is no longer required (please keep in
> mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside
> workspace – this is why it’s done separately!). I just have to fix the Jenkins-
> specific targets in our build.xml in Git repo to only do the actual
> bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine.
> Then it’s a no-op on most builds. For now the separate Jenkins job is enough
> to quickly do the bootstrap as workaround.
> >
> > If nobody stops me from committing the following file into SVN:
> > uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> > total 2.9G
> > -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
> >
> > I will do this later this evening and change the Jenkins builds to do an extra
> checkout into the workspace next to Git.
> > I’d suggest to place the folder contents of ~/lucene-data here:
> https://svn.apache.org/repos/asf/lucene/test-data
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Uwe Schindler [mailto:uschindler@apache.org]
> > Sent: Monday, August 15, 2016 12:47 PM
> > To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler'
> <us...@apache.org>
> > Cc: 'builds' <bu...@apache.org>
> > Subject: RE: JDK, Maven, Ant versions have been consolidated.
> >
> > Hi,
> >
> > the test files could be a puppet module.
> >
> > The lucene.build.properties in the Jenkins home dir is a “hardware specific”
> file, as it allows to configure the number of parallel runners and where the
> special test files are – one would not be able to use it on any other jenkins
> slave. If it is missing, defaults are used, which are not good for optimal CPU
> use. We could add them also to the puppet module, but this would make it
> harder for use to change the file easily.
> >
> > All other files are unmodified.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > uschindler@apache.org
> > ASF Member, Apache Lucene PMC / Committer
> > Bremen, Germany
> > http://lucene.apache.org/
> >
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Monday, August 15, 2016 5:36 AM
> > To: Uwe Schindler <us...@apache.org>
> > Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> > Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >
> > I'll make a copy of the jenkins home dir and get started.
> > It might be an idea to add those test files to puppet wdyt?
> > (I see nothing private about them)
> >
> > Gav...
> >
> > On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
> wrote:
> >> Hi Gav,
> >>
> >> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
> >>
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> uschindler@apache.org
> >> ASF Member, Apache Lucene PMC / Committer
> >> Bremen, Germany
> >> http://lucene.apache.org/
> >>
> >> From: Gav [mailto:gmcdonald@apache.org]
> >> Sent: Friday, August 12, 2016 11:18 AM
> >> To: Uwe Schindler <us...@apache.org>
> >> Cc: builds <bu...@apache.org>
> >> Subject: Re: JDK, Maven, Ant versions have been consolidated.
> >>
> >> Hi Uwe,
> >>
> >> Will get on with your vm tomorrow.
> >>
> >> Gav...
> >>
> >> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
> >>> Hi,
> >>>
> >>> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> >>> FATAL: Cannot find executable from the chosen Ant installation "Ant
> 1.8.4"
> >>>
> >>> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home
> dir keep preserved, we are fine with that:
> >>>
> >>> My mail:
> >>>
> >>> ====================================
> >>> Hi Gav,
> >>>
> >>> sorry for responding late – I am currently on vacation!
> >>>
> >>> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
> >>>
> >>> There is one special case for Lucene:
> >>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that are
> used for nightly tests. If the puppet module does not delete those files in
> current ~/jenkins, it is ok. We would still need access to the machine and
> sudo to jenkins.
> >>>
> >>> If this is all fine, go ahead!
> >>>
> >>> Uwe
> >>> =======================================
> >>>
> >>> Uwe
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>> http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>> > -----Original Message-----
> >>> > From: Gav [mailto:gmcdonald@apache.org]
> >>> > Sent: Friday, August 12, 2016 6:43 AM
> >>> > To: builds <bu...@apache.org>
> >>> > Subject: JDK, Maven, Ant versions have been consolidated.
> >>> >
> >>> > HI All,
> >>> >
> >>> > As per previous emails and as documented [1] [2] [3] we have now
> >>> > consolidated the JDK, Maven and Ant install versions.
> >>> >
> >>> > All versions are now installed/managed via puppet and any 'auto-
> install'
> >>> > versions have been removed.
> >>> >
> >>> > At the time of consolidation and removal of versions, over 200 jobs
> were
> >>> > still configured to use deprecated versions despite the emails I've been
> >>> > sending. Those 200+ jobs I have migrated myself to supported
> versions.
> >>> >
> >>> > Any issues with your jobs please do create a new INFRA jira or email
> this
> >>> > list.
> >>> >
> >>> > Thanks,
> >>> >
> >>> > Gav...
> >>> >
> >>> > [1] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> >>> > [2] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> >>> > [3] -
> >>> >
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> >>> > rix
> >>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Steve Rowe <sa...@gmail.com>.
Thanks Uwe for moving this process along.
+1 to make a new SVN dir at https://svn.apache.org/repos/asf/lucene/test-data/ and put the enwiki data file there.
--
Steve
www.lucidworks.com
> On Aug 15, 2016, at 1:47 PM, Uwe Schindler <us...@apache.org> wrote:
>
> Hi,
>
> I thought about the wikipedia test files used by Jenkins: I think I would commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test Jenkins jobs that use them could simply check them out into a separate dir of the workspace. Byt that it is easier for us to update them. If you also think this is fine, we can leave it like that.
>
> The ~jenkins/lucene.build.properties file is (as said before) machine/hardware specific. I’d keep them in Jenkins’ home dir. We may create a puppet module out of it, but this would prevent us from optimizing or changing them quickly per project requirements.
>
> Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-Bootstrap/). In the future, we can add the “ivy-bootstrap” target as dependency to the “ant jenkins” and other targets which are solely triggered by Jenkins, so separately bootstrapping is no longer required (please keep in mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside workspace – this is why it’s done separately!). I just have to fix the Jenkins-specific targets in our build.xml in Git repo to only do the actual bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine. Then it’s a no-op on most builds. For now the separate Jenkins job is enough to quickly do the bootstrap as workaround.
>
> If nobody stops me from committing the following file into SVN:
> uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
> total 2.9G
> -r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
>
> I will do this later this evening and change the Jenkins builds to do an extra checkout into the workspace next to Git.
> I’d suggest to place the folder contents of ~/lucene-data here: https://svn.apache.org/repos/asf/lucene/test-data
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> From: Uwe Schindler [mailto:uschindler@apache.org]
> Sent: Monday, August 15, 2016 12:47 PM
> To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <us...@apache.org>
> Cc: 'builds' <bu...@apache.org>
> Subject: RE: JDK, Maven, Ant versions have been consolidated.
>
> Hi,
>
> the test files could be a puppet module.
>
> The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
>
> All other files are unmodified.
>
> Uwe
>
> -----
> Uwe Schindler
> uschindler@apache.org
> ASF Member, Apache Lucene PMC / Committer
> Bremen, Germany
> http://lucene.apache.org/
>
> From: Gav [mailto:gmcdonald@apache.org]
> Sent: Monday, August 15, 2016 5:36 AM
> To: Uwe Schindler <us...@apache.org>
> Cc: builds <bu...@apache.org>; dev@lucene.apache.org
> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>
> I'll make a copy of the jenkins home dir and get started.
> It might be an idea to add those test files to puppet wdyt?
> (I see nothing private about them)
>
> Gav...
>
> On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org> wrote:
>> Hi Gav,
>>
>> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> uschindler@apache.org
>> ASF Member, Apache Lucene PMC / Committer
>> Bremen, Germany
>> http://lucene.apache.org/
>>
>> From: Gav [mailto:gmcdonald@apache.org]
>> Sent: Friday, August 12, 2016 11:18 AM
>> To: Uwe Schindler <us...@apache.org>
>> Cc: builds <bu...@apache.org>
>> Subject: Re: JDK, Maven, Ant versions have been consolidated.
>>
>> Hi Uwe,
>>
>> Will get on with your vm tomorrow.
>>
>> Gav...
>>
>> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org> wrote:
>>> Hi,
>>>
>>> on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
>>> FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
>>>
>>> See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
>>>
>>> My mail:
>>>
>>> ====================================
>>> Hi Gav,
>>>
>>> sorry for responding late – I am currently on vacation!
>>>
>>> About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
>>>
>>> There is one special case for Lucene:
>>> We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
>>>
>>> If this is all fine, go ahead!
>>>
>>> Uwe
>>> =======================================
>>>
>>> Uwe
>>>
>>> -----
>>> Uwe Schindler
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>> http://www.thetaphi.de
>>> eMail: uwe@thetaphi.de
>>>
>>> > -----Original Message-----
>>> > From: Gav [mailto:gmcdonald@apache.org]
>>> > Sent: Friday, August 12, 2016 6:43 AM
>>> > To: builds <bu...@apache.org>
>>> > Subject: JDK, Maven, Ant versions have been consolidated.
>>> >
>>> > HI All,
>>> >
>>> > As per previous emails and as documented [1] [2] [3] we have now
>>> > consolidated the JDK, Maven and Ant install versions.
>>> >
>>> > All versions are now installed/managed via puppet and any 'auto-install'
>>> > versions have been removed.
>>> >
>>> > At the time of consolidation and removal of versions, over 200 jobs were
>>> > still configured to use deprecated versions despite the emails I've been
>>> > sending. Those 200+ jobs I have migrated myself to supported versions.
>>> >
>>> > Any issues with your jobs please do create a new INFRA jira or email this
>>> > list.
>>> >
>>> > Thanks,
>>> >
>>> > Gav...
>>> >
>>> > [1] -
>>> > https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>>> > [2] -
>>> > https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
>>> > [3] -
>>> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
>>> > rix
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
I thought about the wikipedia test files used by Jenkins: I think I would commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test Jenkins jobs that use them could simply check them out into a separate dir of the workspace. Byt that it is easier for us to update them. If you also think this is fine, we can leave it like that.
The ~jenkins/lucene.build.properties file is (as said before) machine/hardware specific. I’d keep them in Jenkins’ home dir. We may create a puppet module out of it, but this would prevent us from optimizing or changing them quickly per project requirements.
Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-Bootstrap/). In the future, we can add the “ivy-bootstrap” target as dependency to the “ant jenkins” and other targets which are solely triggered by Jenkins, so separately bootstrapping is no longer required (please keep in mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside workspace – this is why it’s done separately!). I just have to fix the Jenkins-specific targets in our build.xml in Git repo to only do the actual bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine. Then it’s a no-op on most builds. For now the separate Jenkins job is enough to quickly do the bootstrap as workaround.
If nobody stops me from committing the following file into SVN:
uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
total 2.9G
-r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
I will do this later this evening and change the Jenkins builds to do an extra checkout into the workspace next to Git.
I’d suggest to place the folder contents of ~/lucene-data here: https://svn.apache.org/repos/asf/lucene/test-data
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Uwe Schindler [mailto:uschindler@apache.org]
Sent: Monday, August 15, 2016 12:47 PM
To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <us...@apache.org>
Cc: 'builds' <bu...@apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
<http://lucene.apache.org/> http://lucene.apache.org/
From: Gav [ <ma...@apache.org> mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler < <ma...@apache.org> uschindler@apache.org>
Cc: builds < <ma...@apache.org> builds@apache.org>; <ma...@lucene.apache.org> dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hallo Gav,
I just triggered a Lucene build. There was only one problem that the “ivy-bootstrap” was not ran before (which places “Apache IVY” in ~/.ant/lib). I manually triggers “ant ivy-bootstrap” once (developers and people who want to build lucene have to do this once – so also Jenkins).
Now the build works!
I will think about how to handle the Ivy bootstrap part (which only needs to be done once), but please don’t do this in puppet! I think if its ran once, no need to do it again? Or does puppet cleanup the home folder from time to time?
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Uwe Schindler [mailto:uschindler@apache.org]
Sent: Monday, August 15, 2016 12:47 PM
To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <us...@apache.org>
Cc: 'builds' <bu...@apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
<http://lucene.apache.org/> http://lucene.apache.org/
From: Gav [ <ma...@apache.org> mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler < <ma...@apache.org> uschindler@apache.org>
Cc: builds < <ma...@apache.org> builds@apache.org>; <ma...@lucene.apache.org> dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hallo Gav,
I just triggered a Lucene build. There was only one problem that the “ivy-bootstrap” was not ran before (which places “Apache IVY” in ~/.ant/lib). I manually triggers “ant ivy-bootstrap” once (developers and people who want to build lucene have to do this once – so also Jenkins).
Now the build works!
I will think about how to handle the Ivy bootstrap part (which only needs to be done once), but please don’t do this in puppet! I think if its ran once, no need to do it again? Or does puppet cleanup the home folder from time to time?
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Uwe Schindler [mailto:uschindler@apache.org]
Sent: Monday, August 15, 2016 12:47 PM
To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <us...@apache.org>
Cc: 'builds' <bu...@apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
<http://lucene.apache.org/> http://lucene.apache.org/
From: Gav [ <ma...@apache.org> mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler < <ma...@apache.org> uschindler@apache.org>
Cc: builds < <ma...@apache.org> builds@apache.org>; <ma...@lucene.apache.org> dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
I thought about the wikipedia test files used by Jenkins: I think I would commit them to SVN inside the Lucene/Solr project’s folder. The Nightly-Test Jenkins jobs that use them could simply check them out into a separate dir of the workspace. Byt that it is easier for us to update them. If you also think this is fine, we can leave it like that.
The ~jenkins/lucene.build.properties file is (as said before) machine/hardware specific. I’d keep them in Jenkins’ home dir. We may create a puppet module out of it, but this would prevent us from optimizing or changing them quickly per project requirements.
Finally the “ant ivy-bootstrap” task was for now separated into a Jenkins job, that can be manually run (https://builds.apache.org/job/Lucene-Ivy-Bootstrap/). In the future, we can add the “ivy-bootstrap” target as dependency to the “ant jenkins” and other targets which are solely triggered by Jenkins, so separately bootstrapping is no longer required (please keep in mind that bootstrapping places the ivy.jar file in ~/.ant/lib, so outside workspace – this is why it’s done separately!). I just have to fix the Jenkins-specific targets in our build.xml in Git repo to only do the actual bootstrapping if Ivy is not installed locally on Developer’s/Jenkins’ machine. Then it’s a no-op on most builds. For now the separate Jenkins job is enough to quickly do the bootstrap as workaround.
If nobody stops me from committing the following file into SVN:
uschindler@lucene1-us-west:/x1/jenkins/lucene-data$ ls -lh
total 2.9G
-r--r--r-- 1 jenkins jenkins 2.9G May 9 2015 enwiki.random.lines.txt
I will do this later this evening and change the Jenkins builds to do an extra checkout into the workspace next to Git.
I’d suggest to place the folder contents of ~/lucene-data here: https://svn.apache.org/repos/asf/lucene/test-data
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Uwe Schindler [mailto:uschindler@apache.org]
Sent: Monday, August 15, 2016 12:47 PM
To: dev@lucene.apache.org; gmcdonald@apache.org; 'Uwe Schindler' <us...@apache.org>
Cc: 'builds' <bu...@apache.org>
Subject: RE: JDK, Maven, Ant versions have been consolidated.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
<http://lucene.apache.org/> http://lucene.apache.org/
From: Gav [ <ma...@apache.org> mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler < <ma...@apache.org> uschindler@apache.org>
Cc: builds < <ma...@apache.org> builds@apache.org>; <ma...@lucene.apache.org> dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler <us...@apache.org>
Cc: builds <bu...@apache.org>; dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
the test files could be a puppet module.
The lucene.build.properties in the Jenkins home dir is a “hardware specific” file, as it allows to configure the number of parallel runners and where the special test files are – one would not be able to use it on any other jenkins slave. If it is missing, defaults are used, which are not good for optimal CPU use. We could add them also to the puppet module, but this would make it harder for use to change the file easily.
All other files are unmodified.
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Monday, August 15, 2016 5:36 AM
To: Uwe Schindler <us...@apache.org>
Cc: builds <bu...@apache.org>; dev@lucene.apache.org
Subject: Re: JDK, Maven, Ant versions have been consolidated.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org <ma...@apache.org>
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <uschindler@apache.org <ma...@apache.org> >
Cc: builds <builds@apache.org <ma...@apache.org> >
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
wrote:
> Hi Gav,
>
>
>
> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Gav [mailto:gmcdonald@apache.org]
> *Sent:* Friday, August 12, 2016 11:18 AM
> *To:* Uwe Schindler <us...@apache.org>
> *Cc:* builds <bu...@apache.org>
> *Subject:* Re: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Hi Uwe,
>
> Will get on with your vm tomorrow.
>
> Gav...
>
>
>
> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
>
> Hi,
>
> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
>
> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home dir
> keep preserved, we are fine with that:
>
> My mail:
>
> ====================================
> Hi Gav,
>
> sorry for responding late – I am currently on vacation!
>
> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
>
> There is one special case for Lucene:
> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that
> are used for nightly tests. If the puppet module does not delete those
> files in current ~/jenkins, it is ok. We would still need access to the
> machine and sudo to jenkins.
>
> If this is all fine, go ahead!
>
> Uwe
> =======================================
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Friday, August 12, 2016 6:43 AM
> > To: builds <bu...@apache.org>
> > Subject: JDK, Maven, Ant versions have been consolidated.
> >
> > HI All,
> >
> > As per previous emails and as documented [1] [2] [3] we have now
> > consolidated the JDK, Maven and Ant install versions.
> >
> > All versions are now installed/managed via puppet and any 'auto-install'
> > versions have been removed.
> >
> > At the time of consolidation and removal of versions, over 200 jobs were
> > still configured to use deprecated versions despite the emails I've been
> > sending. Those 200+ jobs I have migrated myself to supported versions.
> >
> > Any issues with your jobs please do create a new INFRA jira or email this
> > list.
> >
> > Thanks,
> >
> > Gav...
> >
> > [1] -
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > [2] -
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > [3] -
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > rix
>
>
>
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
I'll make a copy of the jenkins home dir and get started.
It might be an idea to add those test files to puppet wdyt?
(I see nothing private about them)
Gav...
On Sat, Aug 13, 2016 at 3:13 AM, Uwe Schindler <us...@apache.org>
wrote:
> Hi Gav,
>
>
>
> I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed
> the mailing list with like 50 mails today. Can you send me a msg when you
> installed the required puppet modules?
>
>
>
> Uwe
>
>
>
> -----
>
> Uwe Schindler
>
> uschindler@apache.org
>
> ASF Member, Apache Lucene PMC / Committer
>
> Bremen, Germany
>
> http://lucene.apache.org/
>
>
>
> *From:* Gav [mailto:gmcdonald@apache.org]
> *Sent:* Friday, August 12, 2016 11:18 AM
> *To:* Uwe Schindler <us...@apache.org>
> *Cc:* builds <bu...@apache.org>
> *Subject:* Re: JDK, Maven, Ant versions have been consolidated.
>
>
>
> Hi Uwe,
>
> Will get on with your vm tomorrow.
>
> Gav...
>
>
>
> On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
> wrote:
>
> Hi,
>
> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
>
> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home dir
> keep preserved, we are fine with that:
>
> My mail:
>
> ====================================
> Hi Gav,
>
> sorry for responding late – I am currently on vacation!
>
> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
>
> There is one special case for Lucene:
> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that
> are used for nightly tests. If the puppet module does not delete those
> files in current ~/jenkins, it is ok. We would still need access to the
> machine and sudo to jenkins.
>
> If this is all fine, go ahead!
>
> Uwe
> =======================================
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Friday, August 12, 2016 6:43 AM
> > To: builds <bu...@apache.org>
> > Subject: JDK, Maven, Ant versions have been consolidated.
> >
> > HI All,
> >
> > As per previous emails and as documented [1] [2] [3] we have now
> > consolidated the JDK, Maven and Ant install versions.
> >
> > All versions are now installed/managed via puppet and any 'auto-install'
> > versions have been removed.
> >
> > At the time of consolidation and removal of versions, over 200 jobs were
> > still configured to use deprecated versions despite the emails I've been
> > sending. Those 200+ jobs I have migrated myself to supported versions.
> >
> > Any issues with your jobs please do create a new INFRA jira or email this
> > list.
> >
> > Thanks,
> >
> > Gav...
> >
> > [1] -
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > [2] -
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > [3] -
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > rix
>
>
>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <us...@apache.org>
Cc: builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi Gav,
I disabled all auto-running Jenkins Jobs of Lucene/Solr as this spammed the mailing list with like 50 mails today. Can you send me a msg when you installed the required puppet modules?
Uwe
-----
Uwe Schindler
uschindler@apache.org
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/
From: Gav [mailto:gmcdonald@apache.org]
Sent: Friday, August 12, 2016 11:18 AM
To: Uwe Schindler <us...@apache.org>
Cc: builds <bu...@apache.org>
Subject: Re: JDK, Maven, Ant versions have been consolidated.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <uschindler@apache.org <ma...@apache.org> > wrote:
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de <ma...@thetaphi.de>
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org <ma...@apache.org> ]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <builds@apache.org <ma...@apache.org> >
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix
Re: JDK, Maven, Ant versions have been consolidated.
Posted by Gav <gm...@apache.org>.
Hi Uwe,
Will get on with your vm tomorrow.
Gav...
On Fri, Aug 12, 2016 at 6:39 PM, Uwe Schindler <us...@apache.org>
wrote:
> Hi,
>
> on the lucene-jenkins machine, the packages are still missing. I think you
> have to install the jenkins puppet module. Here the message:
> FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
>
> See my e-mail from last week about using the "official" jenkins puppet
> module (answer to your question). Unless our customizations in the home dir
> keep preserved, we are fine with that:
>
> My mail:
>
> ====================================
> Hi Gav,
>
> sorry for responding late – I am currently on vacation!
>
> About your questions: In the past we required some extra modules (like
> older Subversion to be compatible with the older Jenkins Checkouts). As we
> are on Git now and the build scripts use JGit to fetch revision numbers and
> check for clean checkouts, this is obsolete. So if you want to install the
> “Jenkins puppet module”, we are fine.
>
> There is one special case for Lucene:
> We have a hardware-specific “lucene.build.properties” file in Jenkin’s
> root and also some additional “test files” (large wikiepdia checkouts) that
> are used for nightly tests. If the puppet module does not delete those
> files in current ~/jenkins, it is ok. We would still need access to the
> machine and sudo to jenkins.
>
> If this is all fine, go ahead!
>
> Uwe
> =======================================
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Gav [mailto:gmcdonald@apache.org]
> > Sent: Friday, August 12, 2016 6:43 AM
> > To: builds <bu...@apache.org>
> > Subject: JDK, Maven, Ant versions have been consolidated.
> >
> > HI All,
> >
> > As per previous emails and as documented [1] [2] [3] we have now
> > consolidated the JDK, Maven and Ant install versions.
> >
> > All versions are now installed/managed via puppet and any 'auto-install'
> > versions have been removed.
> >
> > At the time of consolidation and removal of versions, over 200 jobs were
> > still configured to use deprecated versions despite the emails I've been
> > sending. Those 200+ jobs I have migrated myself to supported versions.
> >
> > Any issues with your jobs please do create a new INFRA jira or email this
> > list.
> >
> > Thanks,
> >
> > Gav...
> >
> > [1] -
> > https://cwiki.apache.org/confluence/display/INFRA/JDK+
> Installation+Matrix
> > [2] -
> > https://cwiki.apache.org/confluence/display/INFRA/Ant+
> Installation+Matrix
> > [3] -
> > https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> > rix
>
>
RE: JDK, Maven, Ant versions have been consolidated.
Posted by Uwe Schindler <us...@apache.org>.
Hi,
on the lucene-jenkins machine, the packages are still missing. I think you have to install the jenkins puppet module. Here the message:
FATAL: Cannot find executable from the chosen Ant installation "Ant 1.8.4"
See my e-mail from last week about using the "official" jenkins puppet module (answer to your question). Unless our customizations in the home dir keep preserved, we are fine with that:
My mail:
====================================
Hi Gav,
sorry for responding late – I am currently on vacation!
About your questions: In the past we required some extra modules (like older Subversion to be compatible with the older Jenkins Checkouts). As we are on Git now and the build scripts use JGit to fetch revision numbers and check for clean checkouts, this is obsolete. So if you want to install the “Jenkins puppet module”, we are fine.
There is one special case for Lucene:
We have a hardware-specific “lucene.build.properties” file in Jenkin’s root and also some additional “test files” (large wikiepdia checkouts) that are used for nightly tests. If the puppet module does not delete those files in current ~/jenkins, it is ok. We would still need access to the machine and sudo to jenkins.
If this is all fine, go ahead!
Uwe
=======================================
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Gav [mailto:gmcdonald@apache.org]
> Sent: Friday, August 12, 2016 6:43 AM
> To: builds <bu...@apache.org>
> Subject: JDK, Maven, Ant versions have been consolidated.
>
> HI All,
>
> As per previous emails and as documented [1] [2] [3] we have now
> consolidated the JDK, Maven and Ant install versions.
>
> All versions are now installed/managed via puppet and any 'auto-install'
> versions have been removed.
>
> At the time of consolidation and removal of versions, over 200 jobs were
> still configured to use deprecated versions despite the emails I've been
> sending. Those 200+ jobs I have migrated myself to supported versions.
>
> Any issues with your jobs please do create a new INFRA jira or email this
> list.
>
> Thanks,
>
> Gav...
>
> [1] -
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> [2] -
> https://cwiki.apache.org/confluence/display/INFRA/Ant+Installation+Matrix
> [3] -
> https://cwiki.apache.org/confluence/display/INFRA/Maven+Installation+Mat
> rix