You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Enrico Olivelli <eo...@gmail.com> on 2020/07/16 19:56:21 UTC
Fwd: [IMPORTANT] - Migration to ci-builds.a.o
FYI
---------- Forwarded message ---------
Da: Gavin McDonald <gm...@apache.org>
Date: Gio 16 Lug 2020, 18:33
Subject: [IMPORTANT] - Migration to ci-builds.a.o
To: builds <bu...@apache.org>
Hi All,
This NOTICE is for everyone on builds.apache.org. We are migrating to a new
Cloudbees based Client Master called https://ci-builds.apache.org. The
migrations of all jobs needs to be done before the switch off date of 15th
August 2020, so you have a maximum of 4 weeks.
There is no tool or automated way of migrating your jobs, the
differences in the platforms, the plugins and the setup makes it impossible
to do in a safe way. So, you all need to start creating new jobs on
ci-infra.a.o and then , when you are happy, turn off your old builds on
builds.a.o.
There are currently 4 agents over there ready to take jobs, plus a floating
agent which is shared amongst many masters (more to come). I will migrate
away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
will keep an eye of load across both and adjust accordingly.
If needed, create a ticket on INFRA jira for any issues that crop up, or
email here on builds@a.o. there may be one or two plugins we need to
install/tweak etc.
We will be not using 'Views' at the top level, but rather we will take
advantage of 'Folders Plus' - each project will get its own Folder and have
authorisation access to create/edit jobs etc within that folder. I will
create these folders as you ask for them to start with. This setup allows
for credentials isolation amongst other benefits, including but not limited
to exclusive agents (Controlled Agents) for your own use , should you have
any project targeted donations of agents.
As with other aspects of the ASF, projects can choose to just enable all
committers access to their folder, just ask.
We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
be setting up any forwarding rules or anything like that.
So, please, get started *now *on this so you can be sure we are all
completed before the final cutoff date of 15th August 2020.
Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
tickets.
Hadoop and related projects have their own migration path to follow, same
cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
very well in their new homes.
Lets get going ...
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Answering myself:
Flaky Test Report
> On 2020. Aug 14., at 12:52, Andor Molnar <an...@apache.org> wrote:
>
> Hi devs,
>
> There’s some movement from Apache on Jenkins migration. Github account is now available on the new instance which will be useful for PreCommit builds. I started the tests.
>
> Deadline (15 Aug) is approaching. Is there anything else which has fallen through the cracks and has to be migrated to the new instance?
>
> Andor
>
>
>
>> On 2020. Jul 24., at 19:15, Patrick Hunt <ph...@apache.org> wrote:
>>
>> On Fri, Jul 24, 2020 at 3:05 AM Andor Molnar <an...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is
>>> running here:
>>>
>>> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/
>>>
>>> I use standard Git interface to access the repo, because using GitHub
>>> means that branch discovery has to run through the github API. The problem
>>> with this is that I haven’t specified credentials and anonymous access is
>>> limited by some quota, meaning the discovery process takes ages (1-2 hours).
>>>
>>> Builds are running fine except I see a lot of concurrent builds on master
>>> without any meaningful SCM change. Odd. I will change pollSCM trigger to
>>> run @hourly and will see how it goes.
>>>
>>> What’s outstanding?
>>>
>>> - JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would
>>> you like to run against?
>>>
>>> - GitHub Pull Requests precommit job: I have no idea how to do this, but I
>>> suspect we have to use the GitHub Api for this to work.
>>>
>>>
>> Not sure if this is helpful but I saw it recently:
>>
>>> we need ability to trigger builds on pull request creation/updates which
>>
>>> requires the plug-in below or similar:
>>
>>> https://plugins.jenkins.io/ghprb/
>>
>>
>>
>> That plugin is deprecated and not recommended for use .
>>
>>
>> I have installed the *branch source plugin instead*, hopefully that works
>> for
>>
>> you.
>>
>>
>> If not, let's investigate further what else we can do
>>
>>
>>
>>> - Windows build: there’s no Windows agent available currently in the new
>>> instance.
>>>
>>> - Ant builds: do we need this?
>>>
>>> Please share your thoughts.
>>>
>>> Regards,
>>> Andor
>>>
>>>
>>>
>>>
>>>> On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote:
>>>>
>>>> Hi folks,
>>>>
>>>> PR is available: https://github.com/apache/zookeeper/pull/1409
>>>>
>>>> Andor
>>>>
>>>>
>>>>
>>>> On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote:
>>>>> Created a Jira for the task:
>>>>>
>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3896
>>>>>
>>>>> Andor
>>>>>
>>>>>
>>>>>
>>>>>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
>>>>>>
>>>>>> Where’s that example…?
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha
>>>>>>> scritto:
>>>>>>>
>>>>>>>> Hi Enrico,
>>>>>>>>
>>>>>>>> No worries, I only created a few jobs to make some progress,
>>>>>>>> but feel free
>>>>>>>> to ignore that and do it in a better way. The “View” or
>>>>>>>> “Folder” that I was
>>>>>>>> adding jobs is
>>>>>>>>
>>>>>>>> https://ci-hadoop.apache.org/view/ZooKeeper/
>>>>>>>>
>>>>>>>>
>>>>>>>> Andor
>>>>>>>>
>>>>>>>
>>>>>>> This is an example from Apache Maven project. It is very complex
>>>>>>> because
>>>>>>> tests are in another repo and for lots if other reasons. We just
>>>>>>> have to
>>>>>>> create a simpler file.
>>>>>>>
>>>>>>> If nobody volunteers I can try to spend some time but I won't
>>>>>>> have a fast
>>>>>>> pace these weeks
>>>>>>>
>>>>>>>
>>>>>>> Enrico
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <
>>>>>>>>> eolivelli@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha
>>>>>>>>> scritto:
>>>>>>>>>
>>>>>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <
>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org>
>>>>>>>>>>> ha scritto:
>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <
>>>>>>>>>>>> andor@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hi Pat,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have admin rights in the new system too and
>>>>>>>>>>>>> probably can work on
>>>>>>>>>> this
>>>>>>>>>>>> on
>>>>>>>>>>>>> Monday.
>>>>>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate
>>>>>>>>>>>>> the same jobs on
>>>>>>>>>>> the
>>>>>>>>>>>>> new instance?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Not sure on the exact name/feature but "matrix" is
>>>>>>>>>>>> basically the
>>>>>>>>>> ability
>>>>>>>>>>> to
>>>>>>>>>>>> say "run this build/test across a set of JDK versions"
>>>>>>>>>>>> rather than a
>>>>>>>>>>> single
>>>>>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0,
>>>>>>>>>>>> jdk 1/2/3 you end
>>>>>>>>>> up
>>>>>>>>>>>> with a single job which runs three times, one for each
>>>>>>>>>>>> jdk type and
>>>>>>>>>>>> summarizes the results. I've seen this before, I assume
>>>>>>>>>>>> it's a feature
>>>>>>>>>> of
>>>>>>>>>>>> jenkins itself?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> We should use Jenkins files and this configuration will
>>>>>>>>>>> be easy and
>>>>>>>>>>> committed to git
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The ability to do "gitops" would be amazing!
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes.
>>>>>>>>> But I saw on Slack that Andor already started to create new
>>>>>>>>> jobs.
>>>>>>>>> We should do only one way in order not to waste time.
>>>>>>>>>
>>>>>>>>> Andor can you please share your work?
>>>>>>>>> Thanks for doing it
>>>>>>>>>
>>>>>>>>> Enrico
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Patrick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Enrico
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Patrick
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <
>>>>>>>>>>>>>> phunt@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I updated the job I linked earlier based on what's
>>>>>>>>>>>>>> the latest on
>>>>>>>>>> the
>>>>>>>>>>>>> legacy
>>>>>>>>>>>>>> jenkins. It ran successfully
>>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I didn't replicate every config setting - main gap
>>>>>>>>>>>>>> is the spotbugs
>>>>>>>>>>>>>> post-build, which seems to be missing from the new
>>>>>>>>>>>>>> jenkins plugins.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That's just maven master though. Not sure about the
>>>>>>>>>>>>>> rest. Can we do
>>>>>>>>>>>> more
>>>>>>>>>>>>> of
>>>>>>>>>>>>>> a "matrix" config in the new system vs cloning all
>>>>>>>>>>>>>> the time?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <
>>>>>>>>>>>>>> phunt@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
>>>>>>>>>>>>>>> phunt@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico
>>>>>>>>>>>>>>>> Olivelli <
>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> FYI
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related
>>>>>>>>>>>>>>>> projects have their
>>>>>>>>>>> own
>>>>>>>>>>>>>>>> migration path to follow" - any insight on
>>>>>>>>>>>>>>>> that? We are or are
>>>>>>>>>> not
>>>>>>>>>>>>> lumped
>>>>>>>>>>>>>>>> in? I would assume we are?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This (eventual migration) came up a while back
>>>>>>>>>>>>>>>> on the Hadoop PMC
>>>>>>>>>>> and
>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> volunteered to try for us (ZK). I was never
>>>>>>>>>>>>>>>> able to get it to
>>>>>>>>>>> work, I
>>>>>>>>>>>>>>>> provided feedback to infra but they never got
>>>>>>>>>>>>>>>> back, as such we
>>>>>>>>>>> have
>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> project here that's not working with some basic
>>>>>>>>>>>>>>>> dependencies
>>>>>>>>>>> missing:
>>>>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That said, we can try again. Can we verify
>>>>>>>>>>>>>>>> where ZK is supposed
>>>>>>>>>> to
>>>>>>>>>>>>> land?
>>>>>>>>>>>>>>>> Perhaps we can try to delete and recreate the
>>>>>>>>>>>>>>>> POC job I created
>>>>>>>>>> at
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> link to see if we can get it working?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I see another email thread on the list saying
>>>>>>>>>>>>>>> that we are part of
>>>>>>>>>>> said
>>>>>>>>>>>>>>> "related projects". We are expected to move to
>>>>>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks.
>>>>>>>>>>>>>>> Seems nodes are
>>>>>>>>>>> already
>>>>>>>>>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There are over 400 plugins on the current
>>>>>>>>>>>>>>> builds.apache.org -
>>>>>>>>>> most
>>>>>>>>>>> of
>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> we don't need any more, or are replaced with
>>>>>>>>>>>>>>> different plugins on
>>>>>>>>>>> the
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> system. I expect there may be some plugins we
>>>>>>>>>>>>>>> still need to
>>>>>>>>>> install
>>>>>>>>>>> to
>>>>>>>>>>>>> get
>>>>>>>>>>>>>>> you going again, which is why it is vitally
>>>>>>>>>>>>>>> important that you
>>>>>>>>>> start
>>>>>>>>>>>>>>> testing and migrating your jobs over *now*. You
>>>>>>>>>>>>>>> should all have
>>>>>>>>>>> auth.
>>>>>>>>>>>>>>> Any questions, feel free to email the
>>>>>>>>>> hadoop-migrations@apache.org
>>>>>>>>>>>>> list if
>>>>>>>>>>>>>>> you are one of the projects listed below. The
>>>>>>>>>>>>>>> rest of you, not
>>>>>>>>>>>> listed, a
>>>>>>>>>>>>>>> similar email to this one will be posted for you
>>>>>>>>>>>>>>> shortly on
>>>>>>>>>>>> builds@a.o.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> full details
>>>>>>>>>>>>>>>
>>>>>>>>
>>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-
>>>>>>>>>>>>>>>>> builds.a.o
>>>>>>>>>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This NOTICE is for everyone on
>>>>>>>>>>>>>>>>> builds.apache.org. We are
>>>>>>>>>>> migrating
>>>>>>>>>>>>> to a
>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> Cloudbees based Client Master called
>>>>>>>>>> https://ci-builds.apache.org
>>>>>>>>>>> .
>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>> migrations of all jobs needs to be done
>>>>>>>>>>>>>>>>> before the switch off
>>>>>>>>>> date
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> 15th
>>>>>>>>>>>>>>>>> August 2020, so you have a maximum of 4
>>>>>>>>>>>>>>>>> weeks.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There is no tool or automated way of
>>>>>>>>>>>>>>>>> migrating your jobs, the
>>>>>>>>>>>>>>>>> differences in the platforms, the plugins and
>>>>>>>>>>>>>>>>> the setup makes it
>>>>>>>>>>>>>>>>> impossible
>>>>>>>>>>>>>>>>> to do in a safe way. So, you all need to
>>>>>>>>>>>>>>>>> start creating new jobs
>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy,
>>>>>>>>>>>>>>>>> turn off your old
>>>>>>>>>>> builds
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> builds.a.o.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There are currently 4 agents over there ready
>>>>>>>>>>>>>>>>> to take jobs,
>>>>>>>>>> plus a
>>>>>>>>>>>>>>>>> floating
>>>>>>>>>>>>>>>>> agent which is shared amongst many masters
>>>>>>>>>>>>>>>>> (more to come). I
>>>>>>>>>> will
>>>>>>>>>>>>> migrate
>>>>>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci-
>>>>>>>>>>>>>>>>> builds.a.o every few
>>>>>>>>>>> days,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> will keep an eye of load across both and
>>>>>>>>>>>>>>>>> adjust accordingly.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for
>>>>>>>>>>>>>>>>> any issues that
>>>>>>>>>> crop
>>>>>>>>>>>> up,
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> email here on builds@a.o. there may be one or
>>>>>>>>>>>>>>>>> two plugins we
>>>>>>>>>> need
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> install/tweak etc.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We will be not using 'Views' at the top
>>>>>>>>>>>>>>>>> level, but rather we
>>>>>>>>>> will
>>>>>>>>>>>> take
>>>>>>>>>>>>>>>>> advantage of 'Folders Plus' - each project
>>>>>>>>>>>>>>>>> will get its own
>>>>>>>>>> Folder
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> authorisation access to create/edit jobs etc
>>>>>>>>>>>>>>>>> within that
>>>>>>>>>> folder. I
>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> create these folders as you ask for them to
>>>>>>>>>>>>>>>>> start with. This
>>>>>>>>>> setup
>>>>>>>>>>>>> allows
>>>>>>>>>>>>>>>>> for credentials isolation amongst other
>>>>>>>>>>>>>>>>> benefits, including but
>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for
>>>>>>>>>>>>>>>>> your own use ,
>>>>>>>>>> should
>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As with other aspects of the ASF, projects
>>>>>>>>>>>>>>>>> can choose to just
>>>>>>>>>>> enable
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME
>>>>>>>>>>>>>>>>> to ci-builds.a.o
>>>>>>>>>> but
>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>> be setting up any forwarding rules or
>>>>>>>>>>>>>>>>> anything like that.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So, please, get started *now *on this so you
>>>>>>>>>>>>>>>>> can be sure we are
>>>>>>>>>>> all
>>>>>>>>>>>>>>>>> completed before the final cutoff date of
>>>>>>>>>>>>>>>>> 15th August 2020.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) -
>>>>>>>>>>>>>>>>> ask away and/or
>>>>>>>>>> file
>>>>>>>>>>>>> INFRA
>>>>>>>>>>>>>>>>> tickets.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hadoop and related projects have their own
>>>>>>>>>>>>>>>>> migration path to
>>>>>>>>>>> follow,
>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have
>>>>>>>>>>>>>>>>> already migrated and
>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>>> very well in their new homes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Lets get going ...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *Gavin McDonald*
>>>>>>>>>>>>>>>>> Systems Administrator
>>>>>>>>>>>>>>>>> ASF Infrastructure Team
>>>>
>>>
>>>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi devs,
There’s some movement from Apache on Jenkins migration. Github account is now available on the new instance which will be useful for PreCommit builds. I started the tests.
Deadline (15 Aug) is approaching. Is there anything else which has fallen through the cracks and has to be migrated to the new instance?
Andor
> On 2020. Jul 24., at 19:15, Patrick Hunt <ph...@apache.org> wrote:
>
> On Fri, Jul 24, 2020 at 3:05 AM Andor Molnar <an...@apache.org> wrote:
>
>> Hi,
>>
>> Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is
>> running here:
>>
>> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/
>>
>> I use standard Git interface to access the repo, because using GitHub
>> means that branch discovery has to run through the github API. The problem
>> with this is that I haven’t specified credentials and anonymous access is
>> limited by some quota, meaning the discovery process takes ages (1-2 hours).
>>
>> Builds are running fine except I see a lot of concurrent builds on master
>> without any meaningful SCM change. Odd. I will change pollSCM trigger to
>> run @hourly and will see how it goes.
>>
>> What’s outstanding?
>>
>> - JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would
>> you like to run against?
>>
>> - GitHub Pull Requests precommit job: I have no idea how to do this, but I
>> suspect we have to use the GitHub Api for this to work.
>>
>>
> Not sure if this is helpful but I saw it recently:
>
>> we need ability to trigger builds on pull request creation/updates which
>
>> requires the plug-in below or similar:
>
>> https://plugins.jenkins.io/ghprb/
>
>
>
> That plugin is deprecated and not recommended for use .
>
>
> I have installed the *branch source plugin instead*, hopefully that works
> for
>
> you.
>
>
> If not, let's investigate further what else we can do
>
>
>
>> - Windows build: there’s no Windows agent available currently in the new
>> instance.
>>
>> - Ant builds: do we need this?
>>
>> Please share your thoughts.
>>
>> Regards,
>> Andor
>>
>>
>>
>>
>>> On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote:
>>>
>>> Hi folks,
>>>
>>> PR is available: https://github.com/apache/zookeeper/pull/1409
>>>
>>> Andor
>>>
>>>
>>>
>>> On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote:
>>>> Created a Jira for the task:
>>>>
>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-3896
>>>>
>>>> Andor
>>>>
>>>>
>>>>
>>>>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
>>>>>
>>>>> Where’s that example…?
>>>>>
>>>>>
>>>>>
>>>>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha
>>>>>> scritto:
>>>>>>
>>>>>>> Hi Enrico,
>>>>>>>
>>>>>>> No worries, I only created a few jobs to make some progress,
>>>>>>> but feel free
>>>>>>> to ignore that and do it in a better way. The “View” or
>>>>>>> “Folder” that I was
>>>>>>> adding jobs is
>>>>>>>
>>>>>>> https://ci-hadoop.apache.org/view/ZooKeeper/
>>>>>>>
>>>>>>>
>>>>>>> Andor
>>>>>>>
>>>>>>
>>>>>> This is an example from Apache Maven project. It is very complex
>>>>>> because
>>>>>> tests are in another repo and for lots if other reasons. We just
>>>>>> have to
>>>>>> create a simpler file.
>>>>>>
>>>>>> If nobody volunteers I can try to spend some time but I won't
>>>>>> have a fast
>>>>>> pace these weeks
>>>>>>
>>>>>>
>>>>>> Enrico
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <
>>>>>>>> eolivelli@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha
>>>>>>>> scritto:
>>>>>>>>
>>>>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <
>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org>
>>>>>>>>>> ha scritto:
>>>>>>>>>>
>>>>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <
>>>>>>>>>>> andor@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>> Hi Pat,
>>>>>>>>>>>>
>>>>>>>>>>>> I have admin rights in the new system too and
>>>>>>>>>>>> probably can work on
>>>>>>>>> this
>>>>>>>>>>> on
>>>>>>>>>>>> Monday.
>>>>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate
>>>>>>>>>>>> the same jobs on
>>>>>>>>>> the
>>>>>>>>>>>> new instance?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Not sure on the exact name/feature but "matrix" is
>>>>>>>>>>> basically the
>>>>>>>>> ability
>>>>>>>>>> to
>>>>>>>>>>> say "run this build/test across a set of JDK versions"
>>>>>>>>>>> rather than a
>>>>>>>>>> single
>>>>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0,
>>>>>>>>>>> jdk 1/2/3 you end
>>>>>>>>> up
>>>>>>>>>>> with a single job which runs three times, one for each
>>>>>>>>>>> jdk type and
>>>>>>>>>>> summarizes the results. I've seen this before, I assume
>>>>>>>>>>> it's a feature
>>>>>>>>> of
>>>>>>>>>>> jenkins itself?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> We should use Jenkins files and this configuration will
>>>>>>>>>> be easy and
>>>>>>>>>> committed to git
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The ability to do "gitops" would be amazing!
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes.
>>>>>>>> But I saw on Slack that Andor already started to create new
>>>>>>>> jobs.
>>>>>>>> We should do only one way in order not to waste time.
>>>>>>>>
>>>>>>>> Andor can you please share your work?
>>>>>>>> Thanks for doing it
>>>>>>>>
>>>>>>>> Enrico
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Patrick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Enrico
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Patrick
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Andor
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <
>>>>>>>>>>>>> phunt@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I updated the job I linked earlier based on what's
>>>>>>>>>>>>> the latest on
>>>>>>>>> the
>>>>>>>>>>>> legacy
>>>>>>>>>>>>> jenkins. It ran successfully
>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>>>>>>>>
>>>>>>>>>>>>> I didn't replicate every config setting - main gap
>>>>>>>>>>>>> is the spotbugs
>>>>>>>>>>>>> post-build, which seems to be missing from the new
>>>>>>>>>>>>> jenkins plugins.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That's just maven master though. Not sure about the
>>>>>>>>>>>>> rest. Can we do
>>>>>>>>>>> more
>>>>>>>>>>>> of
>>>>>>>>>>>>> a "matrix" config in the new system vs cloning all
>>>>>>>>>>>>> the time?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <
>>>>>>>>>>>>> phunt@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
>>>>>>>>>>>>>> phunt@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico
>>>>>>>>>>>>>>> Olivelli <
>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> FYI
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related
>>>>>>>>>>>>>>> projects have their
>>>>>>>>>> own
>>>>>>>>>>>>>>> migration path to follow" - any insight on
>>>>>>>>>>>>>>> that? We are or are
>>>>>>>>> not
>>>>>>>>>>>> lumped
>>>>>>>>>>>>>>> in? I would assume we are?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This (eventual migration) came up a while back
>>>>>>>>>>>>>>> on the Hadoop PMC
>>>>>>>>>> and
>>>>>>>>>>> I
>>>>>>>>>>>>>>> volunteered to try for us (ZK). I was never
>>>>>>>>>>>>>>> able to get it to
>>>>>>>>>> work, I
>>>>>>>>>>>>>>> provided feedback to infra but they never got
>>>>>>>>>>>>>>> back, as such we
>>>>>>>>>> have
>>>>>>>>>>> a
>>>>>>>>>>>>>>> project here that's not working with some basic
>>>>>>>>>>>>>>> dependencies
>>>>>>>>>> missing:
>>>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That said, we can try again. Can we verify
>>>>>>>>>>>>>>> where ZK is supposed
>>>>>>>>> to
>>>>>>>>>>>> land?
>>>>>>>>>>>>>>> Perhaps we can try to delete and recreate the
>>>>>>>>>>>>>>> POC job I created
>>>>>>>>> at
>>>>>>>>>>> that
>>>>>>>>>>>>>>> link to see if we can get it working?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I see another email thread on the list saying
>>>>>>>>>>>>>> that we are part of
>>>>>>>>>> said
>>>>>>>>>>>>>> "related projects". We are expected to move to
>>>>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks.
>>>>>>>>>>>>>> Seems nodes are
>>>>>>>>>> already
>>>>>>>>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also this:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are over 400 plugins on the current
>>>>>>>>>>>>>> builds.apache.org -
>>>>>>>>> most
>>>>>>>>>> of
>>>>>>>>>>>> which
>>>>>>>>>>>>>> we don't need any more, or are replaced with
>>>>>>>>>>>>>> different plugins on
>>>>>>>>>> the
>>>>>>>>>>>> new
>>>>>>>>>>>>>> system. I expect there may be some plugins we
>>>>>>>>>>>>>> still need to
>>>>>>>>> install
>>>>>>>>>> to
>>>>>>>>>>>> get
>>>>>>>>>>>>>> you going again, which is why it is vitally
>>>>>>>>>>>>>> important that you
>>>>>>>>> start
>>>>>>>>>>>>>> testing and migrating your jobs over *now*. You
>>>>>>>>>>>>>> should all have
>>>>>>>>>> auth.
>>>>>>>>>>>>>> Any questions, feel free to email the
>>>>>>>>> hadoop-migrations@apache.org
>>>>>>>>>>>> list if
>>>>>>>>>>>>>> you are one of the projects listed below. The
>>>>>>>>>>>>>> rest of you, not
>>>>>>>>>>> listed, a
>>>>>>>>>>>>>> similar email to this one will be posted for you
>>>>>>>>>>>>>> shortly on
>>>>>>>>>>> builds@a.o.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> full details
>>>>>>>>>>>>>>
>>>>>>>
>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-
>>>>>>>>>>>>>>>> builds.a.o
>>>>>>>>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This NOTICE is for everyone on
>>>>>>>>>>>>>>>> builds.apache.org. We are
>>>>>>>>>> migrating
>>>>>>>>>>>> to a
>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>> Cloudbees based Client Master called
>>>>>>>>> https://ci-builds.apache.org
>>>>>>>>>> .
>>>>>>>>>>>> The
>>>>>>>>>>>>>>>> migrations of all jobs needs to be done
>>>>>>>>>>>>>>>> before the switch off
>>>>>>>>> date
>>>>>>>>>>> of
>>>>>>>>>>>>>>>> 15th
>>>>>>>>>>>>>>>> August 2020, so you have a maximum of 4
>>>>>>>>>>>>>>>> weeks.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is no tool or automated way of
>>>>>>>>>>>>>>>> migrating your jobs, the
>>>>>>>>>>>>>>>> differences in the platforms, the plugins and
>>>>>>>>>>>>>>>> the setup makes it
>>>>>>>>>>>>>>>> impossible
>>>>>>>>>>>>>>>> to do in a safe way. So, you all need to
>>>>>>>>>>>>>>>> start creating new jobs
>>>>>>>>>> on
>>>>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy,
>>>>>>>>>>>>>>>> turn off your old
>>>>>>>>>> builds
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> builds.a.o.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There are currently 4 agents over there ready
>>>>>>>>>>>>>>>> to take jobs,
>>>>>>>>> plus a
>>>>>>>>>>>>>>>> floating
>>>>>>>>>>>>>>>> agent which is shared amongst many masters
>>>>>>>>>>>>>>>> (more to come). I
>>>>>>>>> will
>>>>>>>>>>>> migrate
>>>>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci-
>>>>>>>>>>>>>>>> builds.a.o every few
>>>>>>>>>> days,
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> will keep an eye of load across both and
>>>>>>>>>>>>>>>> adjust accordingly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for
>>>>>>>>>>>>>>>> any issues that
>>>>>>>>> crop
>>>>>>>>>>> up,
>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> email here on builds@a.o. there may be one or
>>>>>>>>>>>>>>>> two plugins we
>>>>>>>>> need
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> install/tweak etc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We will be not using 'Views' at the top
>>>>>>>>>>>>>>>> level, but rather we
>>>>>>>>> will
>>>>>>>>>>> take
>>>>>>>>>>>>>>>> advantage of 'Folders Plus' - each project
>>>>>>>>>>>>>>>> will get its own
>>>>>>>>> Folder
>>>>>>>>>>> and
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> authorisation access to create/edit jobs etc
>>>>>>>>>>>>>>>> within that
>>>>>>>>> folder. I
>>>>>>>>>>>> will
>>>>>>>>>>>>>>>> create these folders as you ask for them to
>>>>>>>>>>>>>>>> start with. This
>>>>>>>>> setup
>>>>>>>>>>>> allows
>>>>>>>>>>>>>>>> for credentials isolation amongst other
>>>>>>>>>>>>>>>> benefits, including but
>>>>>>>>>> not
>>>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for
>>>>>>>>>>>>>>>> your own use ,
>>>>>>>>> should
>>>>>>>>>>> you
>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As with other aspects of the ASF, projects
>>>>>>>>>>>>>>>> can choose to just
>>>>>>>>>> enable
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME
>>>>>>>>>>>>>>>> to ci-builds.a.o
>>>>>>>>> but
>>>>>>>>>>> will
>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>> be setting up any forwarding rules or
>>>>>>>>>>>>>>>> anything like that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So, please, get started *now *on this so you
>>>>>>>>>>>>>>>> can be sure we are
>>>>>>>>>> all
>>>>>>>>>>>>>>>> completed before the final cutoff date of
>>>>>>>>>>>>>>>> 15th August 2020.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) -
>>>>>>>>>>>>>>>> ask away and/or
>>>>>>>>> file
>>>>>>>>>>>> INFRA
>>>>>>>>>>>>>>>> tickets.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hadoop and related projects have their own
>>>>>>>>>>>>>>>> migration path to
>>>>>>>>>> follow,
>>>>>>>>>>>> same
>>>>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have
>>>>>>>>>>>>>>>> already migrated and
>>>>>>>>>> are
>>>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>>>> very well in their new homes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Lets get going ...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Gavin McDonald*
>>>>>>>>>>>>>>>> Systems Administrator
>>>>>>>>>>>>>>>> ASF Infrastructure Team
>>>
>>
>>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
On Fri, Jul 24, 2020 at 3:05 AM Andor Molnar <an...@apache.org> wrote:
> Hi,
>
> Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is
> running here:
>
> https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/
>
> I use standard Git interface to access the repo, because using GitHub
> means that branch discovery has to run through the github API. The problem
> with this is that I haven’t specified credentials and anonymous access is
> limited by some quota, meaning the discovery process takes ages (1-2 hours).
>
> Builds are running fine except I see a lot of concurrent builds on master
> without any meaningful SCM change. Odd. I will change pollSCM trigger to
> run @hourly and will see how it goes.
>
> What’s outstanding?
>
> - JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would
> you like to run against?
>
> - GitHub Pull Requests precommit job: I have no idea how to do this, but I
> suspect we have to use the GitHub Api for this to work.
>
>
Not sure if this is helpful but I saw it recently:
> we need ability to trigger builds on pull request creation/updates which
> requires the plug-in below or similar:
> https://plugins.jenkins.io/ghprb/
That plugin is deprecated and not recommended for use .
I have installed the *branch source plugin instead*, hopefully that works
for
you.
If not, let's investigate further what else we can do
> - Windows build: there’s no Windows agent available currently in the new
> instance.
>
> - Ant builds: do we need this?
>
> Please share your thoughts.
>
> Regards,
> Andor
>
>
>
>
> > On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote:
> >
> > Hi folks,
> >
> > PR is available: https://github.com/apache/zookeeper/pull/1409
> >
> > Andor
> >
> >
> >
> > On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote:
> >> Created a Jira for the task:
> >>
> >> https://issues.apache.org/jira/browse/ZOOKEEPER-3896
> >>
> >> Andor
> >>
> >>
> >>
> >>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
> >>>
> >>> Where’s that example…?
> >>>
> >>>
> >>>
> >>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha
> >>>> scritto:
> >>>>
> >>>>> Hi Enrico,
> >>>>>
> >>>>> No worries, I only created a few jobs to make some progress,
> >>>>> but feel free
> >>>>> to ignore that and do it in a better way. The “View” or
> >>>>> “Folder” that I was
> >>>>> adding jobs is
> >>>>>
> >>>>> https://ci-hadoop.apache.org/view/ZooKeeper/
> >>>>>
> >>>>>
> >>>>> Andor
> >>>>>
> >>>>
> >>>> This is an example from Apache Maven project. It is very complex
> >>>> because
> >>>> tests are in another repo and for lots if other reasons. We just
> >>>> have to
> >>>> create a simpler file.
> >>>>
> >>>> If nobody volunteers I can try to spend some time but I won't
> >>>> have a fast
> >>>> pace these weeks
> >>>>
> >>>>
> >>>> Enrico
> >>>>
> >>>>
> >>>>
> >>>>>
> >>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <
> >>>>>> eolivelli@gmail.com> wrote:
> >>>>>>
> >>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha
> >>>>>> scritto:
> >>>>>>
> >>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <
> >>>>>>> eolivelli@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org>
> >>>>>>>> ha scritto:
> >>>>>>>>
> >>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <
> >>>>>>>>> andor@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>> Hi Pat,
> >>>>>>>>>>
> >>>>>>>>>> I have admin rights in the new system too and
> >>>>>>>>>> probably can work on
> >>>>>>> this
> >>>>>>>>> on
> >>>>>>>>>> Monday.
> >>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate
> >>>>>>>>>> the same jobs on
> >>>>>>>> the
> >>>>>>>>>> new instance?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>> Not sure on the exact name/feature but "matrix" is
> >>>>>>>>> basically the
> >>>>>>> ability
> >>>>>>>> to
> >>>>>>>>> say "run this build/test across a set of JDK versions"
> >>>>>>>>> rather than a
> >>>>>>>> single
> >>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0,
> >>>>>>>>> jdk 1/2/3 you end
> >>>>>>> up
> >>>>>>>>> with a single job which runs three times, one for each
> >>>>>>>>> jdk type and
> >>>>>>>>> summarizes the results. I've seen this before, I assume
> >>>>>>>>> it's a feature
> >>>>>>> of
> >>>>>>>>> jenkins itself?
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> We should use Jenkins files and this configuration will
> >>>>>>>> be easy and
> >>>>>>>> committed to git
> >>>>>>>>
> >>>>>>>
> >>>>>>> The ability to do "gitops" would be amazing!
> >>>>>>>
> >>>>>>
> >>>>>> Yes.
> >>>>>> But I saw on Slack that Andor already started to create new
> >>>>>> jobs.
> >>>>>> We should do only one way in order not to waste time.
> >>>>>>
> >>>>>> Andor can you please share your work?
> >>>>>> Thanks for doing it
> >>>>>>
> >>>>>> Enrico
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Patrick
> >>>>>>>
> >>>>>>>
> >>>>>>>> Enrico
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Patrick
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Andor
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <
> >>>>>>>>>>> phunt@apache.org> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> I updated the job I linked earlier based on what's
> >>>>>>>>>>> the latest on
> >>>>>>> the
> >>>>>>>>>> legacy
> >>>>>>>>>>> jenkins. It ran successfully
> >>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> >>>>>>>>>>>
> >>>>>>>>>>> I didn't replicate every config setting - main gap
> >>>>>>>>>>> is the spotbugs
> >>>>>>>>>>> post-build, which seems to be missing from the new
> >>>>>>>>>>> jenkins plugins.
> >>>>>>>>>>>
> >>>>>>>>>>> That's just maven master though. Not sure about the
> >>>>>>>>>>> rest. Can we do
> >>>>>>>>> more
> >>>>>>>>>> of
> >>>>>>>>>>> a "matrix" config in the new system vs cloning all
> >>>>>>>>>>> the time?
> >>>>>>>>>>>
> >>>>>>>>>>> Patrick
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <
> >>>>>>>>>>> phunt@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
> >>>>>>>>>>>> phunt@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico
> >>>>>>>>>>>>> Olivelli <
> >>>>>>>>> eolivelli@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> FYI
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related
> >>>>>>>>>>>>> projects have their
> >>>>>>>> own
> >>>>>>>>>>>>> migration path to follow" - any insight on
> >>>>>>>>>>>>> that? We are or are
> >>>>>>> not
> >>>>>>>>>> lumped
> >>>>>>>>>>>>> in? I would assume we are?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> This (eventual migration) came up a while back
> >>>>>>>>>>>>> on the Hadoop PMC
> >>>>>>>> and
> >>>>>>>>> I
> >>>>>>>>>>>>> volunteered to try for us (ZK). I was never
> >>>>>>>>>>>>> able to get it to
> >>>>>>>> work, I
> >>>>>>>>>>>>> provided feedback to infra but they never got
> >>>>>>>>>>>>> back, as such we
> >>>>>>>> have
> >>>>>>>>> a
> >>>>>>>>>>>>> project here that's not working with some basic
> >>>>>>>>>>>>> dependencies
> >>>>>>>> missing:
> >>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That said, we can try again. Can we verify
> >>>>>>>>>>>>> where ZK is supposed
> >>>>>>> to
> >>>>>>>>>> land?
> >>>>>>>>>>>>> Perhaps we can try to delete and recreate the
> >>>>>>>>>>>>> POC job I created
> >>>>>>> at
> >>>>>>>>> that
> >>>>>>>>>>>>> link to see if we can get it working?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> I see another email thread on the list saying
> >>>>>>>>>>>> that we are part of
> >>>>>>>> said
> >>>>>>>>>>>> "related projects". We are expected to move to
> >>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks.
> >>>>>>>>>>>> Seems nodes are
> >>>>>>>> already
> >>>>>>>>>>>> being removed/migrated from the "H#" pool.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Also this:
> >>>>>>>>>>>>
> >>>>>>>>>>>> There are over 400 plugins on the current
> >>>>>>>>>>>> builds.apache.org -
> >>>>>>> most
> >>>>>>>> of
> >>>>>>>>>> which
> >>>>>>>>>>>> we don't need any more, or are replaced with
> >>>>>>>>>>>> different plugins on
> >>>>>>>> the
> >>>>>>>>>> new
> >>>>>>>>>>>> system. I expect there may be some plugins we
> >>>>>>>>>>>> still need to
> >>>>>>> install
> >>>>>>>> to
> >>>>>>>>>> get
> >>>>>>>>>>>> you going again, which is why it is vitally
> >>>>>>>>>>>> important that you
> >>>>>>> start
> >>>>>>>>>>>> testing and migrating your jobs over *now*. You
> >>>>>>>>>>>> should all have
> >>>>>>>> auth.
> >>>>>>>>>>>> Any questions, feel free to email the
> >>>>>>> hadoop-migrations@apache.org
> >>>>>>>>>> list if
> >>>>>>>>>>>> you are one of the projects listed below. The
> >>>>>>>>>>>> rest of you, not
> >>>>>>>>> listed, a
> >>>>>>>>>>>> similar email to this one will be posted for you
> >>>>>>>>>>>> shortly on
> >>>>>>>>> builds@a.o.
> >>>>>>>>>>>>
> >>>>>>>>>>>> full details
> >>>>>>>>>>>>
> >>>>>
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> >>>>>>>>>>>> Patrick
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Patrick
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> ---------- Forwarded message ---------
> >>>>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
> >>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
> >>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-
> >>>>>>>>>>>>>> builds.a.o
> >>>>>>>>>>>>>> To: builds <bu...@apache.org>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This NOTICE is for everyone on
> >>>>>>>>>>>>>> builds.apache.org. We are
> >>>>>>>> migrating
> >>>>>>>>>> to a
> >>>>>>>>>>>>>> new
> >>>>>>>>>>>>>> Cloudbees based Client Master called
> >>>>>>> https://ci-builds.apache.org
> >>>>>>>> .
> >>>>>>>>>> The
> >>>>>>>>>>>>>> migrations of all jobs needs to be done
> >>>>>>>>>>>>>> before the switch off
> >>>>>>> date
> >>>>>>>>> of
> >>>>>>>>>>>>>> 15th
> >>>>>>>>>>>>>> August 2020, so you have a maximum of 4
> >>>>>>>>>>>>>> weeks.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There is no tool or automated way of
> >>>>>>>>>>>>>> migrating your jobs, the
> >>>>>>>>>>>>>> differences in the platforms, the plugins and
> >>>>>>>>>>>>>> the setup makes it
> >>>>>>>>>>>>>> impossible
> >>>>>>>>>>>>>> to do in a safe way. So, you all need to
> >>>>>>>>>>>>>> start creating new jobs
> >>>>>>>> on
> >>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy,
> >>>>>>>>>>>>>> turn off your old
> >>>>>>>> builds
> >>>>>>>>>> on
> >>>>>>>>>>>>>> builds.a.o.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There are currently 4 agents over there ready
> >>>>>>>>>>>>>> to take jobs,
> >>>>>>> plus a
> >>>>>>>>>>>>>> floating
> >>>>>>>>>>>>>> agent which is shared amongst many masters
> >>>>>>>>>>>>>> (more to come). I
> >>>>>>> will
> >>>>>>>>>> migrate
> >>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci-
> >>>>>>>>>>>>>> builds.a.o every few
> >>>>>>>> days,
> >>>>>>>>>> and
> >>>>>>>>>>>>>> will keep an eye of load across both and
> >>>>>>>>>>>>>> adjust accordingly.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for
> >>>>>>>>>>>>>> any issues that
> >>>>>>> crop
> >>>>>>>>> up,
> >>>>>>>>>> or
> >>>>>>>>>>>>>> email here on builds@a.o. there may be one or
> >>>>>>>>>>>>>> two plugins we
> >>>>>>> need
> >>>>>>>>> to
> >>>>>>>>>>>>>> install/tweak etc.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We will be not using 'Views' at the top
> >>>>>>>>>>>>>> level, but rather we
> >>>>>>> will
> >>>>>>>>> take
> >>>>>>>>>>>>>> advantage of 'Folders Plus' - each project
> >>>>>>>>>>>>>> will get its own
> >>>>>>> Folder
> >>>>>>>>> and
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>> authorisation access to create/edit jobs etc
> >>>>>>>>>>>>>> within that
> >>>>>>> folder. I
> >>>>>>>>>> will
> >>>>>>>>>>>>>> create these folders as you ask for them to
> >>>>>>>>>>>>>> start with. This
> >>>>>>> setup
> >>>>>>>>>> allows
> >>>>>>>>>>>>>> for credentials isolation amongst other
> >>>>>>>>>>>>>> benefits, including but
> >>>>>>>> not
> >>>>>>>>>>>>>> limited
> >>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for
> >>>>>>>>>>>>>> your own use ,
> >>>>>>> should
> >>>>>>>>> you
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>>> any project targeted donations of agents.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As with other aspects of the ASF, projects
> >>>>>>>>>>>>>> can choose to just
> >>>>>>>> enable
> >>>>>>>>>> all
> >>>>>>>>>>>>>> committers access to their folder, just ask.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME
> >>>>>>>>>>>>>> to ci-builds.a.o
> >>>>>>> but
> >>>>>>>>> will
> >>>>>>>>>>>>>> not
> >>>>>>>>>>>>>> be setting up any forwarding rules or
> >>>>>>>>>>>>>> anything like that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So, please, get started *now *on this so you
> >>>>>>>>>>>>>> can be sure we are
> >>>>>>>> all
> >>>>>>>>>>>>>> completed before the final cutoff date of
> >>>>>>>>>>>>>> 15th August 2020.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) -
> >>>>>>>>>>>>>> ask away and/or
> >>>>>>> file
> >>>>>>>>>> INFRA
> >>>>>>>>>>>>>> tickets.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hadoop and related projects have their own
> >>>>>>>>>>>>>> migration path to
> >>>>>>>> follow,
> >>>>>>>>>> same
> >>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have
> >>>>>>>>>>>>>> already migrated and
> >>>>>>>> are
> >>>>>>>>>>>>>> doing
> >>>>>>>>>>>>>> very well in their new homes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Lets get going ...
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> *Gavin McDonald*
> >>>>>>>>>>>>>> Systems Administrator
> >>>>>>>>>>>>>> ASF Infrastructure Team
> >
>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi,
Jenkinsfile is now committed to master, 3.5 and 3.6 branches. Build is running here:
https://ci-hadoop.apache.org/view/ZooKeeper/job/zookeeper-master-maven-multipipeline/
I use standard Git interface to access the repo, because using GitHub means that branch discovery has to run through the github API. The problem with this is that I haven’t specified credentials and anonymous access is limited by some quota, meaning the discovery process takes ages (1-2 hours).
Builds are running fine except I see a lot of concurrent builds on master without any meaningful SCM change. Odd. I will change pollSCM trigger to run @hourly and will see how it goes.
What’s outstanding?
- JDKs: currently only two: 8, 11 (LTS versions). What other JDKs would you like to run against?
- GitHub Pull Requests precommit job: I have no idea how to do this, but I suspect we have to use the GitHub Api for this to work.
- Windows build: there’s no Windows agent available currently in the new instance.
- Ant builds: do we need this?
Please share your thoughts.
Regards,
Andor
> On 2020. Jul 23., at 22:08, Andor Molnar <an...@apache.org> wrote:
>
> Hi folks,
>
> PR is available: https://github.com/apache/zookeeper/pull/1409
>
> Andor
>
>
>
> On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote:
>> Created a Jira for the task:
>>
>> https://issues.apache.org/jira/browse/ZOOKEEPER-3896
>>
>> Andor
>>
>>
>>
>>> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
>>>
>>> Where’s that example…?
>>>
>>>
>>>
>>>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com>
>>>> wrote:
>>>>
>>>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha
>>>> scritto:
>>>>
>>>>> Hi Enrico,
>>>>>
>>>>> No worries, I only created a few jobs to make some progress,
>>>>> but feel free
>>>>> to ignore that and do it in a better way. The “View” or
>>>>> “Folder” that I was
>>>>> adding jobs is
>>>>>
>>>>> https://ci-hadoop.apache.org/view/ZooKeeper/
>>>>>
>>>>>
>>>>> Andor
>>>>>
>>>>
>>>> This is an example from Apache Maven project. It is very complex
>>>> because
>>>> tests are in another repo and for lots if other reasons. We just
>>>> have to
>>>> create a simpler file.
>>>>
>>>> If nobody volunteers I can try to spend some time but I won't
>>>> have a fast
>>>> pace these weeks
>>>>
>>>>
>>>> Enrico
>>>>
>>>>
>>>>
>>>>>
>>>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <
>>>>>> eolivelli@gmail.com> wrote:
>>>>>>
>>>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha
>>>>>> scritto:
>>>>>>
>>>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org>
>>>>>>>> ha scritto:
>>>>>>>>
>>>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <
>>>>>>>>> andor@apache.org>
>>>>>>> wrote:
>>>>>>>>>> Hi Pat,
>>>>>>>>>>
>>>>>>>>>> I have admin rights in the new system too and
>>>>>>>>>> probably can work on
>>>>>>> this
>>>>>>>>> on
>>>>>>>>>> Monday.
>>>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate
>>>>>>>>>> the same jobs on
>>>>>>>> the
>>>>>>>>>> new instance?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Not sure on the exact name/feature but "matrix" is
>>>>>>>>> basically the
>>>>>>> ability
>>>>>>>> to
>>>>>>>>> say "run this build/test across a set of JDK versions"
>>>>>>>>> rather than a
>>>>>>>> single
>>>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0,
>>>>>>>>> jdk 1/2/3 you end
>>>>>>> up
>>>>>>>>> with a single job which runs three times, one for each
>>>>>>>>> jdk type and
>>>>>>>>> summarizes the results. I've seen this before, I assume
>>>>>>>>> it's a feature
>>>>>>> of
>>>>>>>>> jenkins itself?
>>>>>>>>>
>>>>>>>>
>>>>>>>> We should use Jenkins files and this configuration will
>>>>>>>> be easy and
>>>>>>>> committed to git
>>>>>>>>
>>>>>>>
>>>>>>> The ability to do "gitops" would be amazing!
>>>>>>>
>>>>>>
>>>>>> Yes.
>>>>>> But I saw on Slack that Andor already started to create new
>>>>>> jobs.
>>>>>> We should do only one way in order not to waste time.
>>>>>>
>>>>>> Andor can you please share your work?
>>>>>> Thanks for doing it
>>>>>>
>>>>>> Enrico
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Patrick
>>>>>>>
>>>>>>>
>>>>>>>> Enrico
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Patrick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Andor
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <
>>>>>>>>>>> phunt@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I updated the job I linked earlier based on what's
>>>>>>>>>>> the latest on
>>>>>>> the
>>>>>>>>>> legacy
>>>>>>>>>>> jenkins. It ran successfully
>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>>>>>>
>>>>>>>>>>> I didn't replicate every config setting - main gap
>>>>>>>>>>> is the spotbugs
>>>>>>>>>>> post-build, which seems to be missing from the new
>>>>>>>>>>> jenkins plugins.
>>>>>>>>>>>
>>>>>>>>>>> That's just maven master though. Not sure about the
>>>>>>>>>>> rest. Can we do
>>>>>>>>> more
>>>>>>>>>> of
>>>>>>>>>>> a "matrix" config in the new system vs cloning all
>>>>>>>>>>> the time?
>>>>>>>>>>>
>>>>>>>>>>> Patrick
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <
>>>>>>>>>>> phunt@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
>>>>>>>>>>>> phunt@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico
>>>>>>>>>>>>> Olivelli <
>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> FYI
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Fun. I do notice it says "Hadoop and related
>>>>>>>>>>>>> projects have their
>>>>>>>> own
>>>>>>>>>>>>> migration path to follow" - any insight on
>>>>>>>>>>>>> that? We are or are
>>>>>>> not
>>>>>>>>>> lumped
>>>>>>>>>>>>> in? I would assume we are?
>>>>>>>>>>>>>
>>>>>>>>>>>>> This (eventual migration) came up a while back
>>>>>>>>>>>>> on the Hadoop PMC
>>>>>>>> and
>>>>>>>>> I
>>>>>>>>>>>>> volunteered to try for us (ZK). I was never
>>>>>>>>>>>>> able to get it to
>>>>>>>> work, I
>>>>>>>>>>>>> provided feedback to infra but they never got
>>>>>>>>>>>>> back, as such we
>>>>>>>> have
>>>>>>>>> a
>>>>>>>>>>>>> project here that's not working with some basic
>>>>>>>>>>>>> dependencies
>>>>>>>> missing:
>>>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>>>>>>
>>>>>>>>>>>>> That said, we can try again. Can we verify
>>>>>>>>>>>>> where ZK is supposed
>>>>>>> to
>>>>>>>>>> land?
>>>>>>>>>>>>> Perhaps we can try to delete and recreate the
>>>>>>>>>>>>> POC job I created
>>>>>>> at
>>>>>>>>> that
>>>>>>>>>>>>> link to see if we can get it working?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> I see another email thread on the list saying
>>>>>>>>>>>> that we are part of
>>>>>>>> said
>>>>>>>>>>>> "related projects". We are expected to move to
>>>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks.
>>>>>>>>>>>> Seems nodes are
>>>>>>>> already
>>>>>>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>>>>>>
>>>>>>>>>>>> Also this:
>>>>>>>>>>>>
>>>>>>>>>>>> There are over 400 plugins on the current
>>>>>>>>>>>> builds.apache.org -
>>>>>>> most
>>>>>>>> of
>>>>>>>>>> which
>>>>>>>>>>>> we don't need any more, or are replaced with
>>>>>>>>>>>> different plugins on
>>>>>>>> the
>>>>>>>>>> new
>>>>>>>>>>>> system. I expect there may be some plugins we
>>>>>>>>>>>> still need to
>>>>>>> install
>>>>>>>> to
>>>>>>>>>> get
>>>>>>>>>>>> you going again, which is why it is vitally
>>>>>>>>>>>> important that you
>>>>>>> start
>>>>>>>>>>>> testing and migrating your jobs over *now*. You
>>>>>>>>>>>> should all have
>>>>>>>> auth.
>>>>>>>>>>>> Any questions, feel free to email the
>>>>>>> hadoop-migrations@apache.org
>>>>>>>>>> list if
>>>>>>>>>>>> you are one of the projects listed below. The
>>>>>>>>>>>> rest of you, not
>>>>>>>>> listed, a
>>>>>>>>>>>> similar email to this one will be posted for you
>>>>>>>>>>>> shortly on
>>>>>>>>> builds@a.o.
>>>>>>>>>>>>
>>>>>>>>>>>> full details
>>>>>>>>>>>>
>>>>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>>>>>> Patrick
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Patrick
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-
>>>>>>>>>>>>>> builds.a.o
>>>>>>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This NOTICE is for everyone on
>>>>>>>>>>>>>> builds.apache.org. We are
>>>>>>>> migrating
>>>>>>>>>> to a
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>> Cloudbees based Client Master called
>>>>>>> https://ci-builds.apache.org
>>>>>>>> .
>>>>>>>>>> The
>>>>>>>>>>>>>> migrations of all jobs needs to be done
>>>>>>>>>>>>>> before the switch off
>>>>>>> date
>>>>>>>>> of
>>>>>>>>>>>>>> 15th
>>>>>>>>>>>>>> August 2020, so you have a maximum of 4
>>>>>>>>>>>>>> weeks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is no tool or automated way of
>>>>>>>>>>>>>> migrating your jobs, the
>>>>>>>>>>>>>> differences in the platforms, the plugins and
>>>>>>>>>>>>>> the setup makes it
>>>>>>>>>>>>>> impossible
>>>>>>>>>>>>>> to do in a safe way. So, you all need to
>>>>>>>>>>>>>> start creating new jobs
>>>>>>>> on
>>>>>>>>>>>>>> ci-infra.a.o and then , when you are happy,
>>>>>>>>>>>>>> turn off your old
>>>>>>>> builds
>>>>>>>>>> on
>>>>>>>>>>>>>> builds.a.o.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are currently 4 agents over there ready
>>>>>>>>>>>>>> to take jobs,
>>>>>>> plus a
>>>>>>>>>>>>>> floating
>>>>>>>>>>>>>> agent which is shared amongst many masters
>>>>>>>>>>>>>> (more to come). I
>>>>>>> will
>>>>>>>>>> migrate
>>>>>>>>>>>>>> away 2 more agents from builds.a.o to ci-
>>>>>>>>>>>>>> builds.a.o every few
>>>>>>>> days,
>>>>>>>>>> and
>>>>>>>>>>>>>> will keep an eye of load across both and
>>>>>>>>>>>>>> adjust accordingly.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If needed, create a ticket on INFRA jira for
>>>>>>>>>>>>>> any issues that
>>>>>>> crop
>>>>>>>>> up,
>>>>>>>>>> or
>>>>>>>>>>>>>> email here on builds@a.o. there may be one or
>>>>>>>>>>>>>> two plugins we
>>>>>>> need
>>>>>>>>> to
>>>>>>>>>>>>>> install/tweak etc.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We will be not using 'Views' at the top
>>>>>>>>>>>>>> level, but rather we
>>>>>>> will
>>>>>>>>> take
>>>>>>>>>>>>>> advantage of 'Folders Plus' - each project
>>>>>>>>>>>>>> will get its own
>>>>>>> Folder
>>>>>>>>> and
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> authorisation access to create/edit jobs etc
>>>>>>>>>>>>>> within that
>>>>>>> folder. I
>>>>>>>>>> will
>>>>>>>>>>>>>> create these folders as you ask for them to
>>>>>>>>>>>>>> start with. This
>>>>>>> setup
>>>>>>>>>> allows
>>>>>>>>>>>>>> for credentials isolation amongst other
>>>>>>>>>>>>>> benefits, including but
>>>>>>>> not
>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>> to exclusive agents (Controlled Agents) for
>>>>>>>>>>>>>> your own use ,
>>>>>>> should
>>>>>>>>> you
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As with other aspects of the ASF, projects
>>>>>>>>>>>>>> can choose to just
>>>>>>>> enable
>>>>>>>>>> all
>>>>>>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME
>>>>>>>>>>>>>> to ci-builds.a.o
>>>>>>> but
>>>>>>>>> will
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> be setting up any forwarding rules or
>>>>>>>>>>>>>> anything like that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, please, get started *now *on this so you
>>>>>>>>>>>>>> can be sure we are
>>>>>>>> all
>>>>>>>>>>>>>> completed before the final cutoff date of
>>>>>>>>>>>>>> 15th August 2020.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) -
>>>>>>>>>>>>>> ask away and/or
>>>>>>> file
>>>>>>>>>> INFRA
>>>>>>>>>>>>>> tickets.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hadoop and related projects have their own
>>>>>>>>>>>>>> migration path to
>>>>>>>> follow,
>>>>>>>>>> same
>>>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have
>>>>>>>>>>>>>> already migrated and
>>>>>>>> are
>>>>>>>>>>>>>> doing
>>>>>>>>>>>>>> very well in their new homes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Lets get going ...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Gavin McDonald*
>>>>>>>>>>>>>> Systems Administrator
>>>>>>>>>>>>>> ASF Infrastructure Team
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi folks,
PR is available: https://github.com/apache/zookeeper/pull/1409
Andor
On Thu, 2020-07-23 at 11:32 +0200, Andor Molnar wrote:
> Created a Jira for the task:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-3896
>
> Andor
>
>
>
> > On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
> >
> > Where’s that example…?
> >
> >
> >
> > > On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha
> > > scritto:
> > >
> > > > Hi Enrico,
> > > >
> > > > No worries, I only created a few jobs to make some progress,
> > > > but feel free
> > > > to ignore that and do it in a better way. The “View” or
> > > > “Folder” that I was
> > > > adding jobs is
> > > >
> > > > https://ci-hadoop.apache.org/view/ZooKeeper/
> > > >
> > > >
> > > > Andor
> > > >
> > >
> > > This is an example from Apache Maven project. It is very complex
> > > because
> > > tests are in another repo and for lots if other reasons. We just
> > > have to
> > > create a simpler file.
> > >
> > > If nobody volunteers I can try to spend some time but I won't
> > > have a fast
> > > pace these weeks
> > >
> > >
> > > Enrico
> > >
> > >
> > >
> > > >
> > > > > On 2020. Jul 20., at 19:34, Enrico Olivelli <
> > > > > eolivelli@gmail.com> wrote:
> > > > >
> > > > > Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha
> > > > > scritto:
> > > > >
> > > > > > On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <
> > > > > > eolivelli@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org>
> > > > > > > ha scritto:
> > > > > > >
> > > > > > > > On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <
> > > > > > > > andor@apache.org>
> > > > > > wrote:
> > > > > > > > > Hi Pat,
> > > > > > > > >
> > > > > > > > > I have admin rights in the new system too and
> > > > > > > > > probably can work on
> > > > > > this
> > > > > > > > on
> > > > > > > > > Monday.
> > > > > > > > > What’s “matrix” config? Shouldn’t we just replicate
> > > > > > > > > the same jobs on
> > > > > > > the
> > > > > > > > > new instance?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Not sure on the exact name/feature but "matrix" is
> > > > > > > > basically the
> > > > > > ability
> > > > > > > to
> > > > > > > > say "run this build/test across a set of JDK versions"
> > > > > > > > rather than a
> > > > > > > single
> > > > > > > > version. As a result, instead of 3 jobs for zk3.6.0,
> > > > > > > > jdk 1/2/3 you end
> > > > > > up
> > > > > > > > with a single job which runs three times, one for each
> > > > > > > > jdk type and
> > > > > > > > summarizes the results. I've seen this before, I assume
> > > > > > > > it's a feature
> > > > > > of
> > > > > > > > jenkins itself?
> > > > > > > >
> > > > > > >
> > > > > > > We should use Jenkins files and this configuration will
> > > > > > > be easy and
> > > > > > > committed to git
> > > > > > >
> > > > > >
> > > > > > The ability to do "gitops" would be amazing!
> > > > > >
> > > > >
> > > > > Yes.
> > > > > But I saw on Slack that Andor already started to create new
> > > > > jobs.
> > > > > We should do only one way in order not to waste time.
> > > > >
> > > > > Andor can you please share your work?
> > > > > Thanks for doing it
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > > > Patrick
> > > > > >
> > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Patrick
> > > > > > > >
> > > > > > > >
> > > > > > > > > Andor
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > On 2020. Jul 17., at 2:51, Patrick Hunt <
> > > > > > > > > > phunt@apache.org> wrote:
> > > > > > > > > >
> > > > > > > > > > I updated the job I linked earlier based on what's
> > > > > > > > > > the latest on
> > > > > > the
> > > > > > > > > legacy
> > > > > > > > > > jenkins. It ran successfully
> > > > > > > > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > > > > > > > > >
> > > > > > > > > > I didn't replicate every config setting - main gap
> > > > > > > > > > is the spotbugs
> > > > > > > > > > post-build, which seems to be missing from the new
> > > > > > > > > > jenkins plugins.
> > > > > > > > > >
> > > > > > > > > > That's just maven master though. Not sure about the
> > > > > > > > > > rest. Can we do
> > > > > > > > more
> > > > > > > > > of
> > > > > > > > > > a "matrix" config in the new system vs cloning all
> > > > > > > > > > the time?
> > > > > > > > > >
> > > > > > > > > > Patrick
> > > > > > > > > >
> > > > > > > > > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <
> > > > > > > > > > phunt@apache.org>
> > > > > > > wrote:
> > > > > > > > > > > On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
> > > > > > > > > > > phunt@apache.org>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Jul 16, 2020 at 12:56 PM Enrico
> > > > > > > > > > > > Olivelli <
> > > > > > > > eolivelli@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > FYI
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Fun. I do notice it says "Hadoop and related
> > > > > > > > > > > > projects have their
> > > > > > > own
> > > > > > > > > > > > migration path to follow" - any insight on
> > > > > > > > > > > > that? We are or are
> > > > > > not
> > > > > > > > > lumped
> > > > > > > > > > > > in? I would assume we are?
> > > > > > > > > > > >
> > > > > > > > > > > > This (eventual migration) came up a while back
> > > > > > > > > > > > on the Hadoop PMC
> > > > > > > and
> > > > > > > > I
> > > > > > > > > > > > volunteered to try for us (ZK). I was never
> > > > > > > > > > > > able to get it to
> > > > > > > work, I
> > > > > > > > > > > > provided feedback to infra but they never got
> > > > > > > > > > > > back, as such we
> > > > > > > have
> > > > > > > > a
> > > > > > > > > > > > project here that's not working with some basic
> > > > > > > > > > > > dependencies
> > > > > > > missing:
> > > > > > > > > > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > > > > > > > > > > >
> > > > > > > > > > > > That said, we can try again. Can we verify
> > > > > > > > > > > > where ZK is supposed
> > > > > > to
> > > > > > > > > land?
> > > > > > > > > > > > Perhaps we can try to delete and recreate the
> > > > > > > > > > > > POC job I created
> > > > > > at
> > > > > > > > that
> > > > > > > > > > > > link to see if we can get it working?
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > I see another email thread on the list saying
> > > > > > > > > > > that we are part of
> > > > > > > said
> > > > > > > > > > > "related projects". We are expected to move to
> > > > > > > > > > > http://ci-hadoop.apache.org/ within 4 weeks.
> > > > > > > > > > > Seems nodes are
> > > > > > > already
> > > > > > > > > > > being removed/migrated from the "H#" pool.
> > > > > > > > > > >
> > > > > > > > > > > Also this:
> > > > > > > > > > >
> > > > > > > > > > > There are over 400 plugins on the current
> > > > > > > > > > > builds.apache.org -
> > > > > > most
> > > > > > > of
> > > > > > > > > which
> > > > > > > > > > > we don't need any more, or are replaced with
> > > > > > > > > > > different plugins on
> > > > > > > the
> > > > > > > > > new
> > > > > > > > > > > system. I expect there may be some plugins we
> > > > > > > > > > > still need to
> > > > > > install
> > > > > > > to
> > > > > > > > > get
> > > > > > > > > > > you going again, which is why it is vitally
> > > > > > > > > > > important that you
> > > > > > start
> > > > > > > > > > > testing and migrating your jobs over *now*. You
> > > > > > > > > > > should all have
> > > > > > > auth.
> > > > > > > > > > > Any questions, feel free to email the
> > > > > > hadoop-migrations@apache.org
> > > > > > > > > list if
> > > > > > > > > > > you are one of the projects listed below. The
> > > > > > > > > > > rest of you, not
> > > > > > > > listed, a
> > > > > > > > > > > similar email to this one will be posted for you
> > > > > > > > > > > shortly on
> > > > > > > > builds@a.o.
> > > > > > > > > > >
> > > > > > > > > > > full details
> > > > > > > > > > >
> > > > https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > > > > > > > > > > Patrick
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > Patrick
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > > ---------- Forwarded message ---------
> > > > > > > > > > > > > Da: Gavin McDonald <gm...@apache.org>
> > > > > > > > > > > > > Date: Gio 16 Lug 2020, 18:33
> > > > > > > > > > > > > Subject: [IMPORTANT] - Migration to ci-
> > > > > > > > > > > > > builds.a.o
> > > > > > > > > > > > > To: builds <bu...@apache.org>
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi All,
> > > > > > > > > > > > >
> > > > > > > > > > > > > This NOTICE is for everyone on
> > > > > > > > > > > > > builds.apache.org. We are
> > > > > > > migrating
> > > > > > > > > to a
> > > > > > > > > > > > > new
> > > > > > > > > > > > > Cloudbees based Client Master called
> > > > > > https://ci-builds.apache.org
> > > > > > > .
> > > > > > > > > The
> > > > > > > > > > > > > migrations of all jobs needs to be done
> > > > > > > > > > > > > before the switch off
> > > > > > date
> > > > > > > > of
> > > > > > > > > > > > > 15th
> > > > > > > > > > > > > August 2020, so you have a maximum of 4
> > > > > > > > > > > > > weeks.
> > > > > > > > > > > > >
> > > > > > > > > > > > > There is no tool or automated way of
> > > > > > > > > > > > > migrating your jobs, the
> > > > > > > > > > > > > differences in the platforms, the plugins and
> > > > > > > > > > > > > the setup makes it
> > > > > > > > > > > > > impossible
> > > > > > > > > > > > > to do in a safe way. So, you all need to
> > > > > > > > > > > > > start creating new jobs
> > > > > > > on
> > > > > > > > > > > > > ci-infra.a.o and then , when you are happy,
> > > > > > > > > > > > > turn off your old
> > > > > > > builds
> > > > > > > > > on
> > > > > > > > > > > > > builds.a.o.
> > > > > > > > > > > > >
> > > > > > > > > > > > > There are currently 4 agents over there ready
> > > > > > > > > > > > > to take jobs,
> > > > > > plus a
> > > > > > > > > > > > > floating
> > > > > > > > > > > > > agent which is shared amongst many masters
> > > > > > > > > > > > > (more to come). I
> > > > > > will
> > > > > > > > > migrate
> > > > > > > > > > > > > away 2 more agents from builds.a.o to ci-
> > > > > > > > > > > > > builds.a.o every few
> > > > > > > days,
> > > > > > > > > and
> > > > > > > > > > > > > will keep an eye of load across both and
> > > > > > > > > > > > > adjust accordingly.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If needed, create a ticket on INFRA jira for
> > > > > > > > > > > > > any issues that
> > > > > > crop
> > > > > > > > up,
> > > > > > > > > or
> > > > > > > > > > > > > email here on builds@a.o. there may be one or
> > > > > > > > > > > > > two plugins we
> > > > > > need
> > > > > > > > to
> > > > > > > > > > > > > install/tweak etc.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We will be not using 'Views' at the top
> > > > > > > > > > > > > level, but rather we
> > > > > > will
> > > > > > > > take
> > > > > > > > > > > > > advantage of 'Folders Plus' - each project
> > > > > > > > > > > > > will get its own
> > > > > > Folder
> > > > > > > > and
> > > > > > > > > > > > > have
> > > > > > > > > > > > > authorisation access to create/edit jobs etc
> > > > > > > > > > > > > within that
> > > > > > folder. I
> > > > > > > > > will
> > > > > > > > > > > > > create these folders as you ask for them to
> > > > > > > > > > > > > start with. This
> > > > > > setup
> > > > > > > > > allows
> > > > > > > > > > > > > for credentials isolation amongst other
> > > > > > > > > > > > > benefits, including but
> > > > > > > not
> > > > > > > > > > > > > limited
> > > > > > > > > > > > > to exclusive agents (Controlled Agents) for
> > > > > > > > > > > > > your own use ,
> > > > > > should
> > > > > > > > you
> > > > > > > > > > > > > have
> > > > > > > > > > > > > any project targeted donations of agents.
> > > > > > > > > > > > >
> > > > > > > > > > > > > As with other aspects of the ASF, projects
> > > > > > > > > > > > > can choose to just
> > > > > > > enable
> > > > > > > > > all
> > > > > > > > > > > > > committers access to their folder, just ask.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We will re-use builds.apache.org as a CNAME
> > > > > > > > > > > > > to ci-builds.a.o
> > > > > > but
> > > > > > > > will
> > > > > > > > > > > > > not
> > > > > > > > > > > > > be setting up any forwarding rules or
> > > > > > > > > > > > > anything like that.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So, please, get started *now *on this so you
> > > > > > > > > > > > > can be sure we are
> > > > > > > all
> > > > > > > > > > > > > completed before the final cutoff date of
> > > > > > > > > > > > > 15th August 2020.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any questions - I expect a few (dozen :) ) -
> > > > > > > > > > > > > ask away and/or
> > > > > > file
> > > > > > > > > INFRA
> > > > > > > > > > > > > tickets.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hadoop and related projects have their own
> > > > > > > > > > > > > migration path to
> > > > > > > follow,
> > > > > > > > > same
> > > > > > > > > > > > > cut off date, Cassandra, Beam, CouchDB have
> > > > > > > > > > > > > already migrated and
> > > > > > > are
> > > > > > > > > > > > > doing
> > > > > > > > > > > > > very well in their new homes.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Lets get going ...
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > *Gavin McDonald*
> > > > > > > > > > > > > Systems Administrator
> > > > > > > > > > > > > ASF Infrastructure Team
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Created a Jira for the task:
https://issues.apache.org/jira/browse/ZOOKEEPER-3896
Andor
> On 2020. Jul 21., at 12:10, Andor Molnar <an...@apache.org> wrote:
>
> Where’s that example…?
>
>
>
>> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com> wrote:
>>
>> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha scritto:
>>
>>> Hi Enrico,
>>>
>>> No worries, I only created a few jobs to make some progress, but feel free
>>> to ignore that and do it in a better way. The “View” or “Folder” that I was
>>> adding jobs is
>>>
>>> https://ci-hadoop.apache.org/view/ZooKeeper/
>>>
>>>
>>> Andor
>>>
>>
>> This is an example from Apache Maven project. It is very complex because
>> tests are in another repo and for lots if other reasons. We just have to
>> create a simpler file.
>>
>> If nobody volunteers I can try to spend some time but I won't have a fast
>> pace these weeks
>>
>>
>> Enrico
>>
>>
>>
>>>
>>>
>>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <eo...@gmail.com> wrote:
>>>>
>>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha scritto:
>>>>
>>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
>>>>>>
>>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org>
>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Pat,
>>>>>>>>
>>>>>>>> I have admin rights in the new system too and probably can work on
>>>>> this
>>>>>>> on
>>>>>>>> Monday.
>>>>>>>> What’s “matrix” config? Shouldn’t we just replicate the same jobs on
>>>>>> the
>>>>>>>> new instance?
>>>>>>>>
>>>>>>>>
>>>>>>> Not sure on the exact name/feature but "matrix" is basically the
>>>>> ability
>>>>>> to
>>>>>>> say "run this build/test across a set of JDK versions" rather than a
>>>>>> single
>>>>>>> version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end
>>>>> up
>>>>>>> with a single job which runs three times, one for each jdk type and
>>>>>>> summarizes the results. I've seen this before, I assume it's a feature
>>>>> of
>>>>>>> jenkins itself?
>>>>>>>
>>>>>>
>>>>>> We should use Jenkins files and this configuration will be easy and
>>>>>> committed to git
>>>>>>
>>>>>
>>>>> The ability to do "gitops" would be amazing!
>>>>>
>>>>
>>>> Yes.
>>>> But I saw on Slack that Andor already started to create new jobs.
>>>> We should do only one way in order not to waste time.
>>>>
>>>> Andor can you please share your work?
>>>> Thanks for doing it
>>>>
>>>> Enrico
>>>>
>>>>
>>>>
>>>>> Patrick
>>>>>
>>>>>
>>>>>>
>>>>>> Enrico
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Patrick
>>>>>>>
>>>>>>>
>>>>>>>> Andor
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> I updated the job I linked earlier based on what's the latest on
>>>>> the
>>>>>>>> legacy
>>>>>>>>> jenkins. It ran successfully
>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>>>>
>>>>>>>>> I didn't replicate every config setting - main gap is the spotbugs
>>>>>>>>> post-build, which seems to be missing from the new jenkins plugins.
>>>>>>>>>
>>>>>>>>> That's just maven master though. Not sure about the rest. Can we do
>>>>>>> more
>>>>>>>> of
>>>>>>>>> a "matrix" config in the new system vs cloning all the time?
>>>>>>>>>
>>>>>>>>> Patrick
>>>>>>>>>
>>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> FYI
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Fun. I do notice it says "Hadoop and related projects have their
>>>>>> own
>>>>>>>>>>> migration path to follow" - any insight on that? We are or are
>>>>> not
>>>>>>>> lumped
>>>>>>>>>>> in? I would assume we are?
>>>>>>>>>>>
>>>>>>>>>>> This (eventual migration) came up a while back on the Hadoop PMC
>>>>>> and
>>>>>>> I
>>>>>>>>>>> volunteered to try for us (ZK). I was never able to get it to
>>>>>> work, I
>>>>>>>>>>> provided feedback to infra but they never got back, as such we
>>>>>> have
>>>>>>> a
>>>>>>>>>>> project here that's not working with some basic dependencies
>>>>>> missing:
>>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>>>>
>>>>>>>>>>> That said, we can try again. Can we verify where ZK is supposed
>>>>> to
>>>>>>>> land?
>>>>>>>>>>> Perhaps we can try to delete and recreate the POC job I created
>>>>> at
>>>>>>> that
>>>>>>>>>>> link to see if we can get it working?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> I see another email thread on the list saying that we are part of
>>>>>> said
>>>>>>>>>> "related projects". We are expected to move to
>>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
>>>>>> already
>>>>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>>>>
>>>>>>>>>> Also this:
>>>>>>>>>>
>>>>>>>>>> There are over 400 plugins on the current builds.apache.org -
>>>>> most
>>>>>> of
>>>>>>>> which
>>>>>>>>>> we don't need any more, or are replaced with different plugins on
>>>>>> the
>>>>>>>> new
>>>>>>>>>> system. I expect there may be some plugins we still need to
>>>>> install
>>>>>> to
>>>>>>>> get
>>>>>>>>>> you going again, which is why it is vitally important that you
>>>>> start
>>>>>>>>>> testing and migrating your jobs over *now*. You should all have
>>>>>> auth.
>>>>>>>>>>
>>>>>>>>>> Any questions, feel free to email the
>>>>> hadoop-migrations@apache.org
>>>>>>>> list if
>>>>>>>>>> you are one of the projects listed below. The rest of you, not
>>>>>>> listed, a
>>>>>>>>>> similar email to this one will be posted for you shortly on
>>>>>>> builds@a.o.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> full details
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>>>>
>>>>>>>>>> Patrick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Patrick
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>>>>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> This NOTICE is for everyone on builds.apache.org. We are
>>>>>> migrating
>>>>>>>> to a
>>>>>>>>>>>> new
>>>>>>>>>>>> Cloudbees based Client Master called
>>>>> https://ci-builds.apache.org
>>>>>> .
>>>>>>>> The
>>>>>>>>>>>> migrations of all jobs needs to be done before the switch off
>>>>> date
>>>>>>> of
>>>>>>>>>>>> 15th
>>>>>>>>>>>> August 2020, so you have a maximum of 4 weeks.
>>>>>>>>>>>>
>>>>>>>>>>>> There is no tool or automated way of migrating your jobs, the
>>>>>>>>>>>> differences in the platforms, the plugins and the setup makes it
>>>>>>>>>>>> impossible
>>>>>>>>>>>> to do in a safe way. So, you all need to start creating new jobs
>>>>>> on
>>>>>>>>>>>> ci-infra.a.o and then , when you are happy, turn off your old
>>>>>> builds
>>>>>>>> on
>>>>>>>>>>>> builds.a.o.
>>>>>>>>>>>>
>>>>>>>>>>>> There are currently 4 agents over there ready to take jobs,
>>>>> plus a
>>>>>>>>>>>> floating
>>>>>>>>>>>> agent which is shared amongst many masters (more to come). I
>>>>> will
>>>>>>>> migrate
>>>>>>>>>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
>>>>>> days,
>>>>>>>> and
>>>>>>>>>>>> will keep an eye of load across both and adjust accordingly.
>>>>>>>>>>>>
>>>>>>>>>>>> If needed, create a ticket on INFRA jira for any issues that
>>>>> crop
>>>>>>> up,
>>>>>>>> or
>>>>>>>>>>>> email here on builds@a.o. there may be one or two plugins we
>>>>> need
>>>>>>> to
>>>>>>>>>>>> install/tweak etc.
>>>>>>>>>>>>
>>>>>>>>>>>> We will be not using 'Views' at the top level, but rather we
>>>>> will
>>>>>>> take
>>>>>>>>>>>> advantage of 'Folders Plus' - each project will get its own
>>>>> Folder
>>>>>>> and
>>>>>>>>>>>> have
>>>>>>>>>>>> authorisation access to create/edit jobs etc within that
>>>>> folder. I
>>>>>>>> will
>>>>>>>>>>>> create these folders as you ask for them to start with. This
>>>>> setup
>>>>>>>> allows
>>>>>>>>>>>> for credentials isolation amongst other benefits, including but
>>>>>> not
>>>>>>>>>>>> limited
>>>>>>>>>>>> to exclusive agents (Controlled Agents) for your own use ,
>>>>> should
>>>>>>> you
>>>>>>>>>>>> have
>>>>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>>>>
>>>>>>>>>>>> As with other aspects of the ASF, projects can choose to just
>>>>>> enable
>>>>>>>> all
>>>>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>>>>
>>>>>>>>>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o
>>>>> but
>>>>>>> will
>>>>>>>>>>>> not
>>>>>>>>>>>> be setting up any forwarding rules or anything like that.
>>>>>>>>>>>>
>>>>>>>>>>>> So, please, get started *now *on this so you can be sure we are
>>>>>> all
>>>>>>>>>>>> completed before the final cutoff date of 15th August 2020.
>>>>>>>>>>>>
>>>>>>>>>>>> Any questions - I expect a few (dozen :) ) - ask away and/or
>>>>> file
>>>>>>>> INFRA
>>>>>>>>>>>> tickets.
>>>>>>>>>>>>
>>>>>>>>>>>> Hadoop and related projects have their own migration path to
>>>>>> follow,
>>>>>>>> same
>>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
>>>>>> are
>>>>>>>>>>>> doing
>>>>>>>>>>>> very well in their new homes.
>>>>>>>>>>>>
>>>>>>>>>>>> Lets get going ...
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>>
>>>>>>>>>>>> *Gavin McDonald*
>>>>>>>>>>>> Systems Administrator
>>>>>>>>>>>> ASF Infrastructure Team
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Where’s that example…?
> On 2020. Jul 20., at 20:02, Enrico Olivelli <eo...@gmail.com> wrote:
>
> Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha scritto:
>
>> Hi Enrico,
>>
>> No worries, I only created a few jobs to make some progress, but feel free
>> to ignore that and do it in a better way. The “View” or “Folder” that I was
>> adding jobs is
>>
>> https://ci-hadoop.apache.org/view/ZooKeeper/
>>
>>
>> Andor
>>
>
> This is an example from Apache Maven project. It is very complex because
> tests are in another repo and for lots if other reasons. We just have to
> create a simpler file.
>
> If nobody volunteers I can try to spend some time but I won't have a fast
> pace these weeks
>
>
> Enrico
>
>
>
>>
>>
>>> On 2020. Jul 20., at 19:34, Enrico Olivelli <eo...@gmail.com> wrote:
>>>
>>> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha scritto:
>>>
>>>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com>
>>>> wrote:
>>>>
>>>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
>>>>>
>>>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> Hi Pat,
>>>>>>>
>>>>>>> I have admin rights in the new system too and probably can work on
>>>> this
>>>>>> on
>>>>>>> Monday.
>>>>>>> What’s “matrix” config? Shouldn’t we just replicate the same jobs on
>>>>> the
>>>>>>> new instance?
>>>>>>>
>>>>>>>
>>>>>> Not sure on the exact name/feature but "matrix" is basically the
>>>> ability
>>>>> to
>>>>>> say "run this build/test across a set of JDK versions" rather than a
>>>>> single
>>>>>> version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end
>>>> up
>>>>>> with a single job which runs three times, one for each jdk type and
>>>>>> summarizes the results. I've seen this before, I assume it's a feature
>>>> of
>>>>>> jenkins itself?
>>>>>>
>>>>>
>>>>> We should use Jenkins files and this configuration will be easy and
>>>>> committed to git
>>>>>
>>>>
>>>> The ability to do "gitops" would be amazing!
>>>>
>>>
>>> Yes.
>>> But I saw on Slack that Andor already started to create new jobs.
>>> We should do only one way in order not to waste time.
>>>
>>> Andor can you please share your work?
>>> Thanks for doing it
>>>
>>> Enrico
>>>
>>>
>>>
>>>> Patrick
>>>>
>>>>
>>>>>
>>>>> Enrico
>>>>>
>>>>>
>>>>>
>>>>>> Patrick
>>>>>>
>>>>>>
>>>>>>> Andor
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
>>>>>>>>
>>>>>>>> I updated the job I linked earlier based on what's the latest on
>>>> the
>>>>>>> legacy
>>>>>>>> jenkins. It ran successfully
>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>>>
>>>>>>>> I didn't replicate every config setting - main gap is the spotbugs
>>>>>>>> post-build, which seems to be missing from the new jenkins plugins.
>>>>>>>>
>>>>>>>> That's just maven master though. Not sure about the rest. Can we do
>>>>>> more
>>>>>>> of
>>>>>>>> a "matrix" config in the new system vs cloning all the time?
>>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
>>>>>> eolivelli@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> FYI
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Fun. I do notice it says "Hadoop and related projects have their
>>>>> own
>>>>>>>>>> migration path to follow" - any insight on that? We are or are
>>>> not
>>>>>>> lumped
>>>>>>>>>> in? I would assume we are?
>>>>>>>>>>
>>>>>>>>>> This (eventual migration) came up a while back on the Hadoop PMC
>>>>> and
>>>>>> I
>>>>>>>>>> volunteered to try for us (ZK). I was never able to get it to
>>>>> work, I
>>>>>>>>>> provided feedback to infra but they never got back, as such we
>>>>> have
>>>>>> a
>>>>>>>>>> project here that's not working with some basic dependencies
>>>>> missing:
>>>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>>>
>>>>>>>>>> That said, we can try again. Can we verify where ZK is supposed
>>>> to
>>>>>>> land?
>>>>>>>>>> Perhaps we can try to delete and recreate the POC job I created
>>>> at
>>>>>> that
>>>>>>>>>> link to see if we can get it working?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I see another email thread on the list saying that we are part of
>>>>> said
>>>>>>>>> "related projects". We are expected to move to
>>>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
>>>>> already
>>>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>>>
>>>>>>>>> Also this:
>>>>>>>>>
>>>>>>>>> There are over 400 plugins on the current builds.apache.org -
>>>> most
>>>>> of
>>>>>>> which
>>>>>>>>> we don't need any more, or are replaced with different plugins on
>>>>> the
>>>>>>> new
>>>>>>>>> system. I expect there may be some plugins we still need to
>>>> install
>>>>> to
>>>>>>> get
>>>>>>>>> you going again, which is why it is vitally important that you
>>>> start
>>>>>>>>> testing and migrating your jobs over *now*. You should all have
>>>>> auth.
>>>>>>>>>
>>>>>>>>> Any questions, feel free to email the
>>>> hadoop-migrations@apache.org
>>>>>>> list if
>>>>>>>>> you are one of the projects listed below. The rest of you, not
>>>>>> listed, a
>>>>>>>>> similar email to this one will be posted for you shortly on
>>>>>> builds@a.o.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> full details
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>>>
>>>>>>>>> Patrick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Patrick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>>>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> This NOTICE is for everyone on builds.apache.org. We are
>>>>> migrating
>>>>>>> to a
>>>>>>>>>>> new
>>>>>>>>>>> Cloudbees based Client Master called
>>>> https://ci-builds.apache.org
>>>>> .
>>>>>>> The
>>>>>>>>>>> migrations of all jobs needs to be done before the switch off
>>>> date
>>>>>> of
>>>>>>>>>>> 15th
>>>>>>>>>>> August 2020, so you have a maximum of 4 weeks.
>>>>>>>>>>>
>>>>>>>>>>> There is no tool or automated way of migrating your jobs, the
>>>>>>>>>>> differences in the platforms, the plugins and the setup makes it
>>>>>>>>>>> impossible
>>>>>>>>>>> to do in a safe way. So, you all need to start creating new jobs
>>>>> on
>>>>>>>>>>> ci-infra.a.o and then , when you are happy, turn off your old
>>>>> builds
>>>>>>> on
>>>>>>>>>>> builds.a.o.
>>>>>>>>>>>
>>>>>>>>>>> There are currently 4 agents over there ready to take jobs,
>>>> plus a
>>>>>>>>>>> floating
>>>>>>>>>>> agent which is shared amongst many masters (more to come). I
>>>> will
>>>>>>> migrate
>>>>>>>>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
>>>>> days,
>>>>>>> and
>>>>>>>>>>> will keep an eye of load across both and adjust accordingly.
>>>>>>>>>>>
>>>>>>>>>>> If needed, create a ticket on INFRA jira for any issues that
>>>> crop
>>>>>> up,
>>>>>>> or
>>>>>>>>>>> email here on builds@a.o. there may be one or two plugins we
>>>> need
>>>>>> to
>>>>>>>>>>> install/tweak etc.
>>>>>>>>>>>
>>>>>>>>>>> We will be not using 'Views' at the top level, but rather we
>>>> will
>>>>>> take
>>>>>>>>>>> advantage of 'Folders Plus' - each project will get its own
>>>> Folder
>>>>>> and
>>>>>>>>>>> have
>>>>>>>>>>> authorisation access to create/edit jobs etc within that
>>>> folder. I
>>>>>>> will
>>>>>>>>>>> create these folders as you ask for them to start with. This
>>>> setup
>>>>>>> allows
>>>>>>>>>>> for credentials isolation amongst other benefits, including but
>>>>> not
>>>>>>>>>>> limited
>>>>>>>>>>> to exclusive agents (Controlled Agents) for your own use ,
>>>> should
>>>>>> you
>>>>>>>>>>> have
>>>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>>>
>>>>>>>>>>> As with other aspects of the ASF, projects can choose to just
>>>>> enable
>>>>>>> all
>>>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>>>
>>>>>>>>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o
>>>> but
>>>>>> will
>>>>>>>>>>> not
>>>>>>>>>>> be setting up any forwarding rules or anything like that.
>>>>>>>>>>>
>>>>>>>>>>> So, please, get started *now *on this so you can be sure we are
>>>>> all
>>>>>>>>>>> completed before the final cutoff date of 15th August 2020.
>>>>>>>>>>>
>>>>>>>>>>> Any questions - I expect a few (dozen :) ) - ask away and/or
>>>> file
>>>>>>> INFRA
>>>>>>>>>>> tickets.
>>>>>>>>>>>
>>>>>>>>>>> Hadoop and related projects have their own migration path to
>>>>> follow,
>>>>>>> same
>>>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
>>>>> are
>>>>>>>>>>> doing
>>>>>>>>>>> very well in their new homes.
>>>>>>>>>>>
>>>>>>>>>>> Lets get going ...
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> *Gavin McDonald*
>>>>>>>>>>> Systems Administrator
>>>>>>>>>>> ASF Infrastructure Team
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Enrico Olivelli <eo...@gmail.com>.
Il Lun 20 Lug 2020, 19:40 Andor Molnar <an...@apache.org> ha scritto:
> Hi Enrico,
>
> No worries, I only created a few jobs to make some progress, but feel free
> to ignore that and do it in a better way. The “View” or “Folder” that I was
> adding jobs is
>
> https://ci-hadoop.apache.org/view/ZooKeeper/
>
>
> Andor
>
This is an example from Apache Maven project. It is very complex because
tests are in another repo and for lots if other reasons. We just have to
create a simpler file.
If nobody volunteers I can try to spend some time but I won't have a fast
pace these weeks
Enrico
>
>
> > On 2020. Jul 20., at 19:34, Enrico Olivelli <eo...@gmail.com> wrote:
> >
> > Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha scritto:
> >
> >> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com>
> >> wrote:
> >>
> >>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
> >>>
> >>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org>
> >> wrote:
> >>>>
> >>>>> Hi Pat,
> >>>>>
> >>>>> I have admin rights in the new system too and probably can work on
> >> this
> >>>> on
> >>>>> Monday.
> >>>>> What’s “matrix” config? Shouldn’t we just replicate the same jobs on
> >>> the
> >>>>> new instance?
> >>>>>
> >>>>>
> >>>> Not sure on the exact name/feature but "matrix" is basically the
> >> ability
> >>> to
> >>>> say "run this build/test across a set of JDK versions" rather than a
> >>> single
> >>>> version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end
> >> up
> >>>> with a single job which runs three times, one for each jdk type and
> >>>> summarizes the results. I've seen this before, I assume it's a feature
> >> of
> >>>> jenkins itself?
> >>>>
> >>>
> >>> We should use Jenkins files and this configuration will be easy and
> >>> committed to git
> >>>
> >>
> >> The ability to do "gitops" would be amazing!
> >>
> >
> > Yes.
> > But I saw on Slack that Andor already started to create new jobs.
> > We should do only one way in order not to waste time.
> >
> > Andor can you please share your work?
> > Thanks for doing it
> >
> > Enrico
> >
> >
> >
> >> Patrick
> >>
> >>
> >>>
> >>> Enrico
> >>>
> >>>
> >>>
> >>>> Patrick
> >>>>
> >>>>
> >>>>> Andor
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> >>>>>>
> >>>>>> I updated the job I linked earlier based on what's the latest on
> >> the
> >>>>> legacy
> >>>>>> jenkins. It ran successfully
> >>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> >>>>>>
> >>>>>> I didn't replicate every config setting - main gap is the spotbugs
> >>>>>> post-build, which seems to be missing from the new jenkins plugins.
> >>>>>>
> >>>>>> That's just maven master though. Not sure about the rest. Can we do
> >>>> more
> >>>>> of
> >>>>>> a "matrix" config in the new system vs cloning all the time?
> >>>>>>
> >>>>>> Patrick
> >>>>>>
> >>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> >>>> eolivelli@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> FYI
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> Fun. I do notice it says "Hadoop and related projects have their
> >>> own
> >>>>>>>> migration path to follow" - any insight on that? We are or are
> >> not
> >>>>> lumped
> >>>>>>>> in? I would assume we are?
> >>>>>>>>
> >>>>>>>> This (eventual migration) came up a while back on the Hadoop PMC
> >>> and
> >>>> I
> >>>>>>>> volunteered to try for us (ZK). I was never able to get it to
> >>> work, I
> >>>>>>>> provided feedback to infra but they never got back, as such we
> >>> have
> >>>> a
> >>>>>>>> project here that's not working with some basic dependencies
> >>> missing:
> >>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> >>>>>>>>
> >>>>>>>> That said, we can try again. Can we verify where ZK is supposed
> >> to
> >>>>> land?
> >>>>>>>> Perhaps we can try to delete and recreate the POC job I created
> >> at
> >>>> that
> >>>>>>>> link to see if we can get it working?
> >>>>>>>>
> >>>>>>>>
> >>>>>>> I see another email thread on the list saying that we are part of
> >>> said
> >>>>>>> "related projects". We are expected to move to
> >>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
> >>> already
> >>>>>>> being removed/migrated from the "H#" pool.
> >>>>>>>
> >>>>>>> Also this:
> >>>>>>>
> >>>>>>> There are over 400 plugins on the current builds.apache.org -
> >> most
> >>> of
> >>>>> which
> >>>>>>> we don't need any more, or are replaced with different plugins on
> >>> the
> >>>>> new
> >>>>>>> system. I expect there may be some plugins we still need to
> >> install
> >>> to
> >>>>> get
> >>>>>>> you going again, which is why it is vitally important that you
> >> start
> >>>>>>> testing and migrating your jobs over *now*. You should all have
> >>> auth.
> >>>>>>>
> >>>>>>> Any questions, feel free to email the
> >> hadoop-migrations@apache.org
> >>>>> list if
> >>>>>>> you are one of the projects listed below. The rest of you, not
> >>>> listed, a
> >>>>>>> similar email to this one will be posted for you shortly on
> >>>> builds@a.o.
> >>>>>>>
> >>>>>>>
> >>>>>>> full details
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> >>>>>>>
> >>>>>>> Patrick
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> Patrick
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ---------- Forwarded message ---------
> >>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
> >>>>>>>>> Date: Gio 16 Lug 2020, 18:33
> >>>>>>>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> >>>>>>>>> To: builds <bu...@apache.org>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Hi All,
> >>>>>>>>>
> >>>>>>>>> This NOTICE is for everyone on builds.apache.org. We are
> >>> migrating
> >>>>> to a
> >>>>>>>>> new
> >>>>>>>>> Cloudbees based Client Master called
> >> https://ci-builds.apache.org
> >>> .
> >>>>> The
> >>>>>>>>> migrations of all jobs needs to be done before the switch off
> >> date
> >>>> of
> >>>>>>>>> 15th
> >>>>>>>>> August 2020, so you have a maximum of 4 weeks.
> >>>>>>>>>
> >>>>>>>>> There is no tool or automated way of migrating your jobs, the
> >>>>>>>>> differences in the platforms, the plugins and the setup makes it
> >>>>>>>>> impossible
> >>>>>>>>> to do in a safe way. So, you all need to start creating new jobs
> >>> on
> >>>>>>>>> ci-infra.a.o and then , when you are happy, turn off your old
> >>> builds
> >>>>> on
> >>>>>>>>> builds.a.o.
> >>>>>>>>>
> >>>>>>>>> There are currently 4 agents over there ready to take jobs,
> >> plus a
> >>>>>>>>> floating
> >>>>>>>>> agent which is shared amongst many masters (more to come). I
> >> will
> >>>>> migrate
> >>>>>>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
> >>> days,
> >>>>> and
> >>>>>>>>> will keep an eye of load across both and adjust accordingly.
> >>>>>>>>>
> >>>>>>>>> If needed, create a ticket on INFRA jira for any issues that
> >> crop
> >>>> up,
> >>>>> or
> >>>>>>>>> email here on builds@a.o. there may be one or two plugins we
> >> need
> >>>> to
> >>>>>>>>> install/tweak etc.
> >>>>>>>>>
> >>>>>>>>> We will be not using 'Views' at the top level, but rather we
> >> will
> >>>> take
> >>>>>>>>> advantage of 'Folders Plus' - each project will get its own
> >> Folder
> >>>> and
> >>>>>>>>> have
> >>>>>>>>> authorisation access to create/edit jobs etc within that
> >> folder. I
> >>>>> will
> >>>>>>>>> create these folders as you ask for them to start with. This
> >> setup
> >>>>> allows
> >>>>>>>>> for credentials isolation amongst other benefits, including but
> >>> not
> >>>>>>>>> limited
> >>>>>>>>> to exclusive agents (Controlled Agents) for your own use ,
> >> should
> >>>> you
> >>>>>>>>> have
> >>>>>>>>> any project targeted donations of agents.
> >>>>>>>>>
> >>>>>>>>> As with other aspects of the ASF, projects can choose to just
> >>> enable
> >>>>> all
> >>>>>>>>> committers access to their folder, just ask.
> >>>>>>>>>
> >>>>>>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o
> >> but
> >>>> will
> >>>>>>>>> not
> >>>>>>>>> be setting up any forwarding rules or anything like that.
> >>>>>>>>>
> >>>>>>>>> So, please, get started *now *on this so you can be sure we are
> >>> all
> >>>>>>>>> completed before the final cutoff date of 15th August 2020.
> >>>>>>>>>
> >>>>>>>>> Any questions - I expect a few (dozen :) ) - ask away and/or
> >> file
> >>>>> INFRA
> >>>>>>>>> tickets.
> >>>>>>>>>
> >>>>>>>>> Hadoop and related projects have their own migration path to
> >>> follow,
> >>>>> same
> >>>>>>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
> >>> are
> >>>>>>>>> doing
> >>>>>>>>> very well in their new homes.
> >>>>>>>>>
> >>>>>>>>> Lets get going ...
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>>
> >>>>>>>>> *Gavin McDonald*
> >>>>>>>>> Systems Administrator
> >>>>>>>>> ASF Infrastructure Team
>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi Enrico,
No worries, I only created a few jobs to make some progress, but feel free to ignore that and do it in a better way. The “View” or “Folder” that I was adding jobs is
https://ci-hadoop.apache.org/view/ZooKeeper/
Andor
> On 2020. Jul 20., at 19:34, Enrico Olivelli <eo...@gmail.com> wrote:
>
> Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha scritto:
>
>> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com>
>> wrote:
>>
>>> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
>>>
>>>> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org>
>> wrote:
>>>>
>>>>> Hi Pat,
>>>>>
>>>>> I have admin rights in the new system too and probably can work on
>> this
>>>> on
>>>>> Monday.
>>>>> What’s “matrix” config? Shouldn’t we just replicate the same jobs on
>>> the
>>>>> new instance?
>>>>>
>>>>>
>>>> Not sure on the exact name/feature but "matrix" is basically the
>> ability
>>> to
>>>> say "run this build/test across a set of JDK versions" rather than a
>>> single
>>>> version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end
>> up
>>>> with a single job which runs three times, one for each jdk type and
>>>> summarizes the results. I've seen this before, I assume it's a feature
>> of
>>>> jenkins itself?
>>>>
>>>
>>> We should use Jenkins files and this configuration will be easy and
>>> committed to git
>>>
>>
>> The ability to do "gitops" would be amazing!
>>
>
> Yes.
> But I saw on Slack that Andor already started to create new jobs.
> We should do only one way in order not to waste time.
>
> Andor can you please share your work?
> Thanks for doing it
>
> Enrico
>
>
>
>> Patrick
>>
>>
>>>
>>> Enrico
>>>
>>>
>>>
>>>> Patrick
>>>>
>>>>
>>>>> Andor
>>>>>
>>>>>
>>>>>
>>>>>> On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
>>>>>>
>>>>>> I updated the job I linked earlier based on what's the latest on
>> the
>>>>> legacy
>>>>>> jenkins. It ran successfully
>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>>>>>>
>>>>>> I didn't replicate every config setting - main gap is the spotbugs
>>>>>> post-build, which seems to be missing from the new jenkins plugins.
>>>>>>
>>>>>> That's just maven master though. Not sure about the rest. Can we do
>>>> more
>>>>> of
>>>>>> a "matrix" config in the new system vs cloning all the time?
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
>>>> eolivelli@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> FYI
>>>>>>>>>
>>>>>>>>
>>>>>>>> Fun. I do notice it says "Hadoop and related projects have their
>>> own
>>>>>>>> migration path to follow" - any insight on that? We are or are
>> not
>>>>> lumped
>>>>>>>> in? I would assume we are?
>>>>>>>>
>>>>>>>> This (eventual migration) came up a while back on the Hadoop PMC
>>> and
>>>> I
>>>>>>>> volunteered to try for us (ZK). I was never able to get it to
>>> work, I
>>>>>>>> provided feedback to infra but they never got back, as such we
>>> have
>>>> a
>>>>>>>> project here that's not working with some basic dependencies
>>> missing:
>>>>>>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>>>>>>
>>>>>>>> That said, we can try again. Can we verify where ZK is supposed
>> to
>>>>> land?
>>>>>>>> Perhaps we can try to delete and recreate the POC job I created
>> at
>>>> that
>>>>>>>> link to see if we can get it working?
>>>>>>>>
>>>>>>>>
>>>>>>> I see another email thread on the list saying that we are part of
>>> said
>>>>>>> "related projects". We are expected to move to
>>>>>>> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
>>> already
>>>>>>> being removed/migrated from the "H#" pool.
>>>>>>>
>>>>>>> Also this:
>>>>>>>
>>>>>>> There are over 400 plugins on the current builds.apache.org -
>> most
>>> of
>>>>> which
>>>>>>> we don't need any more, or are replaced with different plugins on
>>> the
>>>>> new
>>>>>>> system. I expect there may be some plugins we still need to
>> install
>>> to
>>>>> get
>>>>>>> you going again, which is why it is vitally important that you
>> start
>>>>>>> testing and migrating your jobs over *now*. You should all have
>>> auth.
>>>>>>>
>>>>>>> Any questions, feel free to email the
>> hadoop-migrations@apache.org
>>>>> list if
>>>>>>> you are one of the projects listed below. The rest of you, not
>>>> listed, a
>>>>>>> similar email to this one will be posted for you shortly on
>>>> builds@a.o.
>>>>>>>
>>>>>>>
>>>>>>> full details
>>>>>>>
>>>>>
>>>>
>>>
>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>>>>>>
>>>>>>> Patrick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------- Forwarded message ---------
>>>>>>>>> Da: Gavin McDonald <gm...@apache.org>
>>>>>>>>> Date: Gio 16 Lug 2020, 18:33
>>>>>>>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>>>>>>>>> To: builds <bu...@apache.org>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> This NOTICE is for everyone on builds.apache.org. We are
>>> migrating
>>>>> to a
>>>>>>>>> new
>>>>>>>>> Cloudbees based Client Master called
>> https://ci-builds.apache.org
>>> .
>>>>> The
>>>>>>>>> migrations of all jobs needs to be done before the switch off
>> date
>>>> of
>>>>>>>>> 15th
>>>>>>>>> August 2020, so you have a maximum of 4 weeks.
>>>>>>>>>
>>>>>>>>> There is no tool or automated way of migrating your jobs, the
>>>>>>>>> differences in the platforms, the plugins and the setup makes it
>>>>>>>>> impossible
>>>>>>>>> to do in a safe way. So, you all need to start creating new jobs
>>> on
>>>>>>>>> ci-infra.a.o and then , when you are happy, turn off your old
>>> builds
>>>>> on
>>>>>>>>> builds.a.o.
>>>>>>>>>
>>>>>>>>> There are currently 4 agents over there ready to take jobs,
>> plus a
>>>>>>>>> floating
>>>>>>>>> agent which is shared amongst many masters (more to come). I
>> will
>>>>> migrate
>>>>>>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
>>> days,
>>>>> and
>>>>>>>>> will keep an eye of load across both and adjust accordingly.
>>>>>>>>>
>>>>>>>>> If needed, create a ticket on INFRA jira for any issues that
>> crop
>>>> up,
>>>>> or
>>>>>>>>> email here on builds@a.o. there may be one or two plugins we
>> need
>>>> to
>>>>>>>>> install/tweak etc.
>>>>>>>>>
>>>>>>>>> We will be not using 'Views' at the top level, but rather we
>> will
>>>> take
>>>>>>>>> advantage of 'Folders Plus' - each project will get its own
>> Folder
>>>> and
>>>>>>>>> have
>>>>>>>>> authorisation access to create/edit jobs etc within that
>> folder. I
>>>>> will
>>>>>>>>> create these folders as you ask for them to start with. This
>> setup
>>>>> allows
>>>>>>>>> for credentials isolation amongst other benefits, including but
>>> not
>>>>>>>>> limited
>>>>>>>>> to exclusive agents (Controlled Agents) for your own use ,
>> should
>>>> you
>>>>>>>>> have
>>>>>>>>> any project targeted donations of agents.
>>>>>>>>>
>>>>>>>>> As with other aspects of the ASF, projects can choose to just
>>> enable
>>>>> all
>>>>>>>>> committers access to their folder, just ask.
>>>>>>>>>
>>>>>>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o
>> but
>>>> will
>>>>>>>>> not
>>>>>>>>> be setting up any forwarding rules or anything like that.
>>>>>>>>>
>>>>>>>>> So, please, get started *now *on this so you can be sure we are
>>> all
>>>>>>>>> completed before the final cutoff date of 15th August 2020.
>>>>>>>>>
>>>>>>>>> Any questions - I expect a few (dozen :) ) - ask away and/or
>> file
>>>>> INFRA
>>>>>>>>> tickets.
>>>>>>>>>
>>>>>>>>> Hadoop and related projects have their own migration path to
>>> follow,
>>>>> same
>>>>>>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
>>> are
>>>>>>>>> doing
>>>>>>>>> very well in their new homes.
>>>>>>>>>
>>>>>>>>> Lets get going ...
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> *Gavin McDonald*
>>>>>>>>> Systems Administrator
>>>>>>>>> ASF Infrastructure Team
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Enrico Olivelli <eo...@gmail.com>.
Il Lun 20 Lug 2020, 19:02 Patrick Hunt <ph...@apache.org> ha scritto:
> On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
> >
> > > On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org>
> wrote:
> > >
> > > > Hi Pat,
> > > >
> > > > I have admin rights in the new system too and probably can work on
> this
> > > on
> > > > Monday.
> > > > What’s “matrix” config? Shouldn’t we just replicate the same jobs on
> > the
> > > > new instance?
> > > >
> > > >
> > > Not sure on the exact name/feature but "matrix" is basically the
> ability
> > to
> > > say "run this build/test across a set of JDK versions" rather than a
> > single
> > > version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end
> up
> > > with a single job which runs three times, one for each jdk type and
> > > summarizes the results. I've seen this before, I assume it's a feature
> of
> > > jenkins itself?
> > >
> >
> > We should use Jenkins files and this configuration will be easy and
> > committed to git
> >
>
> The ability to do "gitops" would be amazing!
>
Yes.
But I saw on Slack that Andor already started to create new jobs.
We should do only one way in order not to waste time.
Andor can you please share your work?
Thanks for doing it
Enrico
> Patrick
>
>
> >
> > Enrico
> >
> >
> >
> > > Patrick
> > >
> > >
> > > > Andor
> > > >
> > > >
> > > >
> > > > > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> > > > >
> > > > > I updated the job I linked earlier based on what's the latest on
> the
> > > > legacy
> > > > > jenkins. It ran successfully
> > > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > > > >
> > > > > I didn't replicate every config setting - main gap is the spotbugs
> > > > > post-build, which seems to be missing from the new jenkins plugins.
> > > > >
> > > > > That's just maven master though. Not sure about the rest. Can we do
> > > more
> > > > of
> > > > > a "matrix" config in the new system vs cloning all the time?
> > > > >
> > > > > Patrick
> > > > >
> > > > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
> > wrote:
> > > > >
> > > > >>
> > > > >> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
> > > wrote:
> > > > >>
> > > > >>>
> > > > >>>
> > > > >>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> > > eolivelli@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> FYI
> > > > >>>>
> > > > >>>
> > > > >>> Fun. I do notice it says "Hadoop and related projects have their
> > own
> > > > >>> migration path to follow" - any insight on that? We are or are
> not
> > > > lumped
> > > > >>> in? I would assume we are?
> > > > >>>
> > > > >>> This (eventual migration) came up a while back on the Hadoop PMC
> > and
> > > I
> > > > >>> volunteered to try for us (ZK). I was never able to get it to
> > work, I
> > > > >>> provided feedback to infra but they never got back, as such we
> > have
> > > a
> > > > >>> project here that's not working with some basic dependencies
> > missing:
> > > > >>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > > > >>>
> > > > >>> That said, we can try again. Can we verify where ZK is supposed
> to
> > > > land?
> > > > >>> Perhaps we can try to delete and recreate the POC job I created
> at
> > > that
> > > > >>> link to see if we can get it working?
> > > > >>>
> > > > >>>
> > > > >> I see another email thread on the list saying that we are part of
> > said
> > > > >> "related projects". We are expected to move to
> > > > >> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
> > already
> > > > >> being removed/migrated from the "H#" pool.
> > > > >>
> > > > >> Also this:
> > > > >>
> > > > >> There are over 400 plugins on the current builds.apache.org -
> most
> > of
> > > > which
> > > > >> we don't need any more, or are replaced with different plugins on
> > the
> > > > new
> > > > >> system. I expect there may be some plugins we still need to
> install
> > to
> > > > get
> > > > >> you going again, which is why it is vitally important that you
> start
> > > > >> testing and migrating your jobs over *now*. You should all have
> > auth.
> > > > >>
> > > > >> Any questions, feel free to email the
> hadoop-migrations@apache.org
> > > > list if
> > > > >> you are one of the projects listed below. The rest of you, not
> > > listed, a
> > > > >> similar email to this one will be posted for you shortly on
> > > builds@a.o.
> > > > >>
> > > > >>
> > > > >> full details
> > > > >>
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > > > >>
> > > > >> Patrick
> > > > >>
> > > > >>
> > > > >>
> > > > >>> Patrick
> > > > >>>
> > > > >>>
> > > > >>>>
> > > > >>>> ---------- Forwarded message ---------
> > > > >>>> Da: Gavin McDonald <gm...@apache.org>
> > > > >>>> Date: Gio 16 Lug 2020, 18:33
> > > > >>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> > > > >>>> To: builds <bu...@apache.org>
> > > > >>>>
> > > > >>>>
> > > > >>>> Hi All,
> > > > >>>>
> > > > >>>> This NOTICE is for everyone on builds.apache.org. We are
> > migrating
> > > > to a
> > > > >>>> new
> > > > >>>> Cloudbees based Client Master called
> https://ci-builds.apache.org
> > .
> > > > The
> > > > >>>> migrations of all jobs needs to be done before the switch off
> date
> > > of
> > > > >>>> 15th
> > > > >>>> August 2020, so you have a maximum of 4 weeks.
> > > > >>>>
> > > > >>>> There is no tool or automated way of migrating your jobs, the
> > > > >>>> differences in the platforms, the plugins and the setup makes it
> > > > >>>> impossible
> > > > >>>> to do in a safe way. So, you all need to start creating new jobs
> > on
> > > > >>>> ci-infra.a.o and then , when you are happy, turn off your old
> > builds
> > > > on
> > > > >>>> builds.a.o.
> > > > >>>>
> > > > >>>> There are currently 4 agents over there ready to take jobs,
> plus a
> > > > >>>> floating
> > > > >>>> agent which is shared amongst many masters (more to come). I
> will
> > > > migrate
> > > > >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
> > days,
> > > > and
> > > > >>>> will keep an eye of load across both and adjust accordingly.
> > > > >>>>
> > > > >>>> If needed, create a ticket on INFRA jira for any issues that
> crop
> > > up,
> > > > or
> > > > >>>> email here on builds@a.o. there may be one or two plugins we
> need
> > > to
> > > > >>>> install/tweak etc.
> > > > >>>>
> > > > >>>> We will be not using 'Views' at the top level, but rather we
> will
> > > take
> > > > >>>> advantage of 'Folders Plus' - each project will get its own
> Folder
> > > and
> > > > >>>> have
> > > > >>>> authorisation access to create/edit jobs etc within that
> folder. I
> > > > will
> > > > >>>> create these folders as you ask for them to start with. This
> setup
> > > > allows
> > > > >>>> for credentials isolation amongst other benefits, including but
> > not
> > > > >>>> limited
> > > > >>>> to exclusive agents (Controlled Agents) for your own use ,
> should
> > > you
> > > > >>>> have
> > > > >>>> any project targeted donations of agents.
> > > > >>>>
> > > > >>>> As with other aspects of the ASF, projects can choose to just
> > enable
> > > > all
> > > > >>>> committers access to their folder, just ask.
> > > > >>>>
> > > > >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o
> but
> > > will
> > > > >>>> not
> > > > >>>> be setting up any forwarding rules or anything like that.
> > > > >>>>
> > > > >>>> So, please, get started *now *on this so you can be sure we are
> > all
> > > > >>>> completed before the final cutoff date of 15th August 2020.
> > > > >>>>
> > > > >>>> Any questions - I expect a few (dozen :) ) - ask away and/or
> file
> > > > INFRA
> > > > >>>> tickets.
> > > > >>>>
> > > > >>>> Hadoop and related projects have their own migration path to
> > follow,
> > > > same
> > > > >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
> > are
> > > > >>>> doing
> > > > >>>> very well in their new homes.
> > > > >>>>
> > > > >>>> Lets get going ...
> > > > >>>>
> > > > >>>> --
> > > > >>>>
> > > > >>>> *Gavin McDonald*
> > > > >>>> Systems Administrator
> > > > >>>> ASF Infrastructure Team
> > > > >>>>
> > > > >>>
> > > >
> > > >
> > >
> >
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
On Mon, Jul 20, 2020 at 9:47 AM Enrico Olivelli <eo...@gmail.com> wrote:
> Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
>
> > On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org> wrote:
> >
> > > Hi Pat,
> > >
> > > I have admin rights in the new system too and probably can work on this
> > on
> > > Monday.
> > > What’s “matrix” config? Shouldn’t we just replicate the same jobs on
> the
> > > new instance?
> > >
> > >
> > Not sure on the exact name/feature but "matrix" is basically the ability
> to
> > say "run this build/test across a set of JDK versions" rather than a
> single
> > version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end up
> > with a single job which runs three times, one for each jdk type and
> > summarizes the results. I've seen this before, I assume it's a feature of
> > jenkins itself?
> >
>
> We should use Jenkins files and this configuration will be easy and
> committed to git
>
The ability to do "gitops" would be amazing!
Patrick
>
> Enrico
>
>
>
> > Patrick
> >
> >
> > > Andor
> > >
> > >
> > >
> > > > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> > > >
> > > > I updated the job I linked earlier based on what's the latest on the
> > > legacy
> > > > jenkins. It ran successfully
> > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > > >
> > > > I didn't replicate every config setting - main gap is the spotbugs
> > > > post-build, which seems to be missing from the new jenkins plugins.
> > > >
> > > > That's just maven master though. Not sure about the rest. Can we do
> > more
> > > of
> > > > a "matrix" config in the new system vs cloning all the time?
> > > >
> > > > Patrick
> > > >
> > > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
> wrote:
> > > >
> > > >>
> > > >> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
> > wrote:
> > > >>
> > > >>>
> > > >>>
> > > >>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> > eolivelli@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>>> FYI
> > > >>>>
> > > >>>
> > > >>> Fun. I do notice it says "Hadoop and related projects have their
> own
> > > >>> migration path to follow" - any insight on that? We are or are not
> > > lumped
> > > >>> in? I would assume we are?
> > > >>>
> > > >>> This (eventual migration) came up a while back on the Hadoop PMC
> and
> > I
> > > >>> volunteered to try for us (ZK). I was never able to get it to
> work, I
> > > >>> provided feedback to infra but they never got back, as such we
> have
> > a
> > > >>> project here that's not working with some basic dependencies
> missing:
> > > >>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > > >>>
> > > >>> That said, we can try again. Can we verify where ZK is supposed to
> > > land?
> > > >>> Perhaps we can try to delete and recreate the POC job I created at
> > that
> > > >>> link to see if we can get it working?
> > > >>>
> > > >>>
> > > >> I see another email thread on the list saying that we are part of
> said
> > > >> "related projects". We are expected to move to
> > > >> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
> already
> > > >> being removed/migrated from the "H#" pool.
> > > >>
> > > >> Also this:
> > > >>
> > > >> There are over 400 plugins on the current builds.apache.org - most
> of
> > > which
> > > >> we don't need any more, or are replaced with different plugins on
> the
> > > new
> > > >> system. I expect there may be some plugins we still need to install
> to
> > > get
> > > >> you going again, which is why it is vitally important that you start
> > > >> testing and migrating your jobs over *now*. You should all have
> auth.
> > > >>
> > > >> Any questions, feel free to email the hadoop-migrations@apache.org
> > > list if
> > > >> you are one of the projects listed below. The rest of you, not
> > listed, a
> > > >> similar email to this one will be posted for you shortly on
> > builds@a.o.
> > > >>
> > > >>
> > > >> full details
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > > >>
> > > >> Patrick
> > > >>
> > > >>
> > > >>
> > > >>> Patrick
> > > >>>
> > > >>>
> > > >>>>
> > > >>>> ---------- Forwarded message ---------
> > > >>>> Da: Gavin McDonald <gm...@apache.org>
> > > >>>> Date: Gio 16 Lug 2020, 18:33
> > > >>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> > > >>>> To: builds <bu...@apache.org>
> > > >>>>
> > > >>>>
> > > >>>> Hi All,
> > > >>>>
> > > >>>> This NOTICE is for everyone on builds.apache.org. We are
> migrating
> > > to a
> > > >>>> new
> > > >>>> Cloudbees based Client Master called https://ci-builds.apache.org
> .
> > > The
> > > >>>> migrations of all jobs needs to be done before the switch off date
> > of
> > > >>>> 15th
> > > >>>> August 2020, so you have a maximum of 4 weeks.
> > > >>>>
> > > >>>> There is no tool or automated way of migrating your jobs, the
> > > >>>> differences in the platforms, the plugins and the setup makes it
> > > >>>> impossible
> > > >>>> to do in a safe way. So, you all need to start creating new jobs
> on
> > > >>>> ci-infra.a.o and then , when you are happy, turn off your old
> builds
> > > on
> > > >>>> builds.a.o.
> > > >>>>
> > > >>>> There are currently 4 agents over there ready to take jobs, plus a
> > > >>>> floating
> > > >>>> agent which is shared amongst many masters (more to come). I will
> > > migrate
> > > >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few
> days,
> > > and
> > > >>>> will keep an eye of load across both and adjust accordingly.
> > > >>>>
> > > >>>> If needed, create a ticket on INFRA jira for any issues that crop
> > up,
> > > or
> > > >>>> email here on builds@a.o. there may be one or two plugins we need
> > to
> > > >>>> install/tweak etc.
> > > >>>>
> > > >>>> We will be not using 'Views' at the top level, but rather we will
> > take
> > > >>>> advantage of 'Folders Plus' - each project will get its own Folder
> > and
> > > >>>> have
> > > >>>> authorisation access to create/edit jobs etc within that folder. I
> > > will
> > > >>>> create these folders as you ask for them to start with. This setup
> > > allows
> > > >>>> for credentials isolation amongst other benefits, including but
> not
> > > >>>> limited
> > > >>>> to exclusive agents (Controlled Agents) for your own use , should
> > you
> > > >>>> have
> > > >>>> any project targeted donations of agents.
> > > >>>>
> > > >>>> As with other aspects of the ASF, projects can choose to just
> enable
> > > all
> > > >>>> committers access to their folder, just ask.
> > > >>>>
> > > >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but
> > will
> > > >>>> not
> > > >>>> be setting up any forwarding rules or anything like that.
> > > >>>>
> > > >>>> So, please, get started *now *on this so you can be sure we are
> all
> > > >>>> completed before the final cutoff date of 15th August 2020.
> > > >>>>
> > > >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file
> > > INFRA
> > > >>>> tickets.
> > > >>>>
> > > >>>> Hadoop and related projects have their own migration path to
> follow,
> > > same
> > > >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and
> are
> > > >>>> doing
> > > >>>> very well in their new homes.
> > > >>>>
> > > >>>> Lets get going ...
> > > >>>>
> > > >>>> --
> > > >>>>
> > > >>>> *Gavin McDonald*
> > > >>>> Systems Administrator
> > > >>>> ASF Infrastructure Team
> > > >>>>
> > > >>>
> > >
> > >
> >
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Enrico Olivelli <eo...@gmail.com>.
Il Lun 20 Lug 2020, 18:41 Patrick Hunt <ph...@apache.org> ha scritto:
> On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org> wrote:
>
> > Hi Pat,
> >
> > I have admin rights in the new system too and probably can work on this
> on
> > Monday.
> > What’s “matrix” config? Shouldn’t we just replicate the same jobs on the
> > new instance?
> >
> >
> Not sure on the exact name/feature but "matrix" is basically the ability to
> say "run this build/test across a set of JDK versions" rather than a single
> version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end up
> with a single job which runs three times, one for each jdk type and
> summarizes the results. I've seen this before, I assume it's a feature of
> jenkins itself?
>
We should use Jenkins files and this configuration will be easy and
committed to git
Enrico
> Patrick
>
>
> > Andor
> >
> >
> >
> > > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> > >
> > > I updated the job I linked earlier based on what's the latest on the
> > legacy
> > > jenkins. It ran successfully
> > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > >
> > > I didn't replicate every config setting - main gap is the spotbugs
> > > post-build, which seems to be missing from the new jenkins plugins.
> > >
> > > That's just maven master though. Not sure about the rest. Can we do
> more
> > of
> > > a "matrix" config in the new system vs cloning all the time?
> > >
> > > Patrick
> > >
> > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org> wrote:
> > >
> > >>
> > >> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
> wrote:
> > >>
> > >>>
> > >>>
> > >>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> eolivelli@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> FYI
> > >>>>
> > >>>
> > >>> Fun. I do notice it says "Hadoop and related projects have their own
> > >>> migration path to follow" - any insight on that? We are or are not
> > lumped
> > >>> in? I would assume we are?
> > >>>
> > >>> This (eventual migration) came up a while back on the Hadoop PMC and
> I
> > >>> volunteered to try for us (ZK). I was never able to get it to work, I
> > >>> provided feedback to infra but they never got back, as such we have
> a
> > >>> project here that's not working with some basic dependencies missing:
> > >>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > >>>
> > >>> That said, we can try again. Can we verify where ZK is supposed to
> > land?
> > >>> Perhaps we can try to delete and recreate the POC job I created at
> that
> > >>> link to see if we can get it working?
> > >>>
> > >>>
> > >> I see another email thread on the list saying that we are part of said
> > >> "related projects". We are expected to move to
> > >> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are already
> > >> being removed/migrated from the "H#" pool.
> > >>
> > >> Also this:
> > >>
> > >> There are over 400 plugins on the current builds.apache.org - most of
> > which
> > >> we don't need any more, or are replaced with different plugins on the
> > new
> > >> system. I expect there may be some plugins we still need to install to
> > get
> > >> you going again, which is why it is vitally important that you start
> > >> testing and migrating your jobs over *now*. You should all have auth.
> > >>
> > >> Any questions, feel free to email the hadoop-migrations@apache.org
> > list if
> > >> you are one of the projects listed below. The rest of you, not
> listed, a
> > >> similar email to this one will be posted for you shortly on
> builds@a.o.
> > >>
> > >>
> > >> full details
> > >>
> >
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > >>
> > >> Patrick
> > >>
> > >>
> > >>
> > >>> Patrick
> > >>>
> > >>>
> > >>>>
> > >>>> ---------- Forwarded message ---------
> > >>>> Da: Gavin McDonald <gm...@apache.org>
> > >>>> Date: Gio 16 Lug 2020, 18:33
> > >>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> > >>>> To: builds <bu...@apache.org>
> > >>>>
> > >>>>
> > >>>> Hi All,
> > >>>>
> > >>>> This NOTICE is for everyone on builds.apache.org. We are migrating
> > to a
> > >>>> new
> > >>>> Cloudbees based Client Master called https://ci-builds.apache.org.
> > The
> > >>>> migrations of all jobs needs to be done before the switch off date
> of
> > >>>> 15th
> > >>>> August 2020, so you have a maximum of 4 weeks.
> > >>>>
> > >>>> There is no tool or automated way of migrating your jobs, the
> > >>>> differences in the platforms, the plugins and the setup makes it
> > >>>> impossible
> > >>>> to do in a safe way. So, you all need to start creating new jobs on
> > >>>> ci-infra.a.o and then , when you are happy, turn off your old builds
> > on
> > >>>> builds.a.o.
> > >>>>
> > >>>> There are currently 4 agents over there ready to take jobs, plus a
> > >>>> floating
> > >>>> agent which is shared amongst many masters (more to come). I will
> > migrate
> > >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days,
> > and
> > >>>> will keep an eye of load across both and adjust accordingly.
> > >>>>
> > >>>> If needed, create a ticket on INFRA jira for any issues that crop
> up,
> > or
> > >>>> email here on builds@a.o. there may be one or two plugins we need
> to
> > >>>> install/tweak etc.
> > >>>>
> > >>>> We will be not using 'Views' at the top level, but rather we will
> take
> > >>>> advantage of 'Folders Plus' - each project will get its own Folder
> and
> > >>>> have
> > >>>> authorisation access to create/edit jobs etc within that folder. I
> > will
> > >>>> create these folders as you ask for them to start with. This setup
> > allows
> > >>>> for credentials isolation amongst other benefits, including but not
> > >>>> limited
> > >>>> to exclusive agents (Controlled Agents) for your own use , should
> you
> > >>>> have
> > >>>> any project targeted donations of agents.
> > >>>>
> > >>>> As with other aspects of the ASF, projects can choose to just enable
> > all
> > >>>> committers access to their folder, just ask.
> > >>>>
> > >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but
> will
> > >>>> not
> > >>>> be setting up any forwarding rules or anything like that.
> > >>>>
> > >>>> So, please, get started *now *on this so you can be sure we are all
> > >>>> completed before the final cutoff date of 15th August 2020.
> > >>>>
> > >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file
> > INFRA
> > >>>> tickets.
> > >>>>
> > >>>> Hadoop and related projects have their own migration path to follow,
> > same
> > >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
> > >>>> doing
> > >>>> very well in their new homes.
> > >>>>
> > >>>> Lets get going ...
> > >>>>
> > >>>> --
> > >>>>
> > >>>> *Gavin McDonald*
> > >>>> Systems Administrator
> > >>>> ASF Infrastructure Team
> > >>>>
> > >>>
> >
> >
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
On Sat, Jul 18, 2020 at 12:20 PM Andor Molnar <an...@apache.org> wrote:
> Hi Pat,
>
> I have admin rights in the new system too and probably can work on this on
> Monday.
> What’s “matrix” config? Shouldn’t we just replicate the same jobs on the
> new instance?
>
>
Not sure on the exact name/feature but "matrix" is basically the ability to
say "run this build/test across a set of JDK versions" rather than a single
version. As a result, instead of 3 jobs for zk3.6.0, jdk 1/2/3 you end up
with a single job which runs three times, one for each jdk type and
summarizes the results. I've seen this before, I assume it's a feature of
jenkins itself?
Patrick
> Andor
>
>
>
> > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> >
> > I updated the job I linked earlier based on what's the latest on the
> legacy
> > jenkins. It ran successfully
> > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> >
> > I didn't replicate every config setting - main gap is the spotbugs
> > post-build, which seems to be missing from the new jenkins plugins.
> >
> > That's just maven master though. Not sure about the rest. Can we do more
> of
> > a "matrix" config in the new system vs cloning all the time?
> >
> > Patrick
> >
> > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org> wrote:
> >
> >>
> >> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org> wrote:
> >>
> >>>
> >>>
> >>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
> >>> wrote:
> >>>
> >>>> FYI
> >>>>
> >>>
> >>> Fun. I do notice it says "Hadoop and related projects have their own
> >>> migration path to follow" - any insight on that? We are or are not
> lumped
> >>> in? I would assume we are?
> >>>
> >>> This (eventual migration) came up a while back on the Hadoop PMC and I
> >>> volunteered to try for us (ZK). I was never able to get it to work, I
> >>> provided feedback to infra but they never got back, as such we have a
> >>> project here that's not working with some basic dependencies missing:
> >>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> >>>
> >>> That said, we can try again. Can we verify where ZK is supposed to
> land?
> >>> Perhaps we can try to delete and recreate the POC job I created at that
> >>> link to see if we can get it working?
> >>>
> >>>
> >> I see another email thread on the list saying that we are part of said
> >> "related projects". We are expected to move to
> >> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are already
> >> being removed/migrated from the "H#" pool.
> >>
> >> Also this:
> >>
> >> There are over 400 plugins on the current builds.apache.org - most of
> which
> >> we don't need any more, or are replaced with different plugins on the
> new
> >> system. I expect there may be some plugins we still need to install to
> get
> >> you going again, which is why it is vitally important that you start
> >> testing and migrating your jobs over *now*. You should all have auth.
> >>
> >> Any questions, feel free to email the hadoop-migrations@apache.org
> list if
> >> you are one of the projects listed below. The rest of you, not listed, a
> >> similar email to this one will be posted for you shortly on builds@a.o.
> >>
> >>
> >> full details
> >>
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> >>
> >> Patrick
> >>
> >>
> >>
> >>> Patrick
> >>>
> >>>
> >>>>
> >>>> ---------- Forwarded message ---------
> >>>> Da: Gavin McDonald <gm...@apache.org>
> >>>> Date: Gio 16 Lug 2020, 18:33
> >>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> >>>> To: builds <bu...@apache.org>
> >>>>
> >>>>
> >>>> Hi All,
> >>>>
> >>>> This NOTICE is for everyone on builds.apache.org. We are migrating
> to a
> >>>> new
> >>>> Cloudbees based Client Master called https://ci-builds.apache.org.
> The
> >>>> migrations of all jobs needs to be done before the switch off date of
> >>>> 15th
> >>>> August 2020, so you have a maximum of 4 weeks.
> >>>>
> >>>> There is no tool or automated way of migrating your jobs, the
> >>>> differences in the platforms, the plugins and the setup makes it
> >>>> impossible
> >>>> to do in a safe way. So, you all need to start creating new jobs on
> >>>> ci-infra.a.o and then , when you are happy, turn off your old builds
> on
> >>>> builds.a.o.
> >>>>
> >>>> There are currently 4 agents over there ready to take jobs, plus a
> >>>> floating
> >>>> agent which is shared amongst many masters (more to come). I will
> migrate
> >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days,
> and
> >>>> will keep an eye of load across both and adjust accordingly.
> >>>>
> >>>> If needed, create a ticket on INFRA jira for any issues that crop up,
> or
> >>>> email here on builds@a.o. there may be one or two plugins we need to
> >>>> install/tweak etc.
> >>>>
> >>>> We will be not using 'Views' at the top level, but rather we will take
> >>>> advantage of 'Folders Plus' - each project will get its own Folder and
> >>>> have
> >>>> authorisation access to create/edit jobs etc within that folder. I
> will
> >>>> create these folders as you ask for them to start with. This setup
> allows
> >>>> for credentials isolation amongst other benefits, including but not
> >>>> limited
> >>>> to exclusive agents (Controlled Agents) for your own use , should you
> >>>> have
> >>>> any project targeted donations of agents.
> >>>>
> >>>> As with other aspects of the ASF, projects can choose to just enable
> all
> >>>> committers access to their folder, just ask.
> >>>>
> >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> >>>> not
> >>>> be setting up any forwarding rules or anything like that.
> >>>>
> >>>> So, please, get started *now *on this so you can be sure we are all
> >>>> completed before the final cutoff date of 15th August 2020.
> >>>>
> >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file
> INFRA
> >>>> tickets.
> >>>>
> >>>> Hadoop and related projects have their own migration path to follow,
> same
> >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
> >>>> doing
> >>>> very well in their new homes.
> >>>>
> >>>> Lets get going ...
> >>>>
> >>>> --
> >>>>
> >>>> *Gavin McDonald*
> >>>> Systems Administrator
> >>>> ASF Infrastructure Team
> >>>>
> >>>
>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
There's one thing I noticed is that the summary of each build is
missing from the build's status page. It's probably caused by a missing
Jenkins plugin, but I don't know. Build preferences are roughly the
same.
Andor
On Mon, 2020-07-20 at 15:48 +0200, Andor Molnar wrote:
> Hi,
>
> I created a few new jobs based on your suggestions. Before going
> forward please take a look at them.
>
> I created Jdk LTS jobs (8 and 11) for all active branches: 3.5, 3.6
> and
> master. Also added the Maven option surefire.forkcount=4 to all jobs
> to
> limit the number of concurrent threads in unit tests.
>
> Do you think we also need non-LTS Java build jobs too?
>
> https://ci-hadoop.apache.org/view/ZooKeeper/
>
> Andor
>
>
>
> On Sun, 2020-07-19 at 07:50 +0200, Enrico Olivelli wrote:
> > Il Sab 18 Lug 2020, 21:20 Andor Molnar <an...@apache.org> ha
> > scritto:
> >
> > > Hi Pat,
> > >
> > > I have admin rights in the new system too and probably can work
> > > on
> > > this on
> > > Monday.
> > > What’s “matrix” config? Shouldn’t we just replicate the same jobs
> > > on the
> > > new instance?
> > >
> >
> > This is my list of important jobs:
> > - maven builds of master against every supported jdk +EA (for JDK
> > outreach
> > quality program)
> > - maven build of master on windows on jdk8
> > - maven build of branch-3.5 and branch-3.6
> > - ant build of branch-3.5 on jdk8
> > - maven precommit job
> >
> > Enrico
> >
> >
> > > Andor
> > >
> > >
> > >
> > > > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org>
> > > > wrote:
> > > >
> > > > I updated the job I linked earlier based on what's the latest
> > > > on
> > > > the
> > > legacy
> > > > jenkins. It ran successfully
> > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > > >
> > > > I didn't replicate every config setting - main gap is the
> > > > spotbugs
> > > > post-build, which seems to be missing from the new jenkins
> > > > plugins.
> > > >
> > > > That's just maven master though. Not sure about the rest. Can
> > > > we
> > > > do more
> > > of
> > > > a "matrix" config in the new system vs cloning all the time?
> > > >
> > > > Patrick
> > > >
> > > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
> > > > wrote:
> > > >
> > > > > On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <
> > > > > phunt@apache.org>
> > > > > wrote:
> > > > >
> > > > > > On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> > > > > > eolivelli@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > FYI
> > > > > > >
> > > > > >
> > > > > > Fun. I do notice it says "Hadoop and related projects have
> > > > > > their own
> > > > > > migration path to follow" - any insight on that? We are or
> > > > > > are not
> > > lumped
> > > > > > in? I would assume we are?
> > > > > >
> > > > > > This (eventual migration) came up a while back on the
> > > > > > Hadoop
> > > > > > PMC and I
> > > > > > volunteered to try for us (ZK). I was never able to get it
> > > > > > to
> > > > > > work, I
> > > > > > provided feedback to infra but they never got back, as
> > > > > > such
> > > > > > we have a
> > > > > > project here that's not working with some basic
> > > > > > dependencies
> > > > > > missing:
> > > > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > > > > >
> > > > > > That said, we can try again. Can we verify where ZK is
> > > > > > supposed to
> > > land?
> > > > > > Perhaps we can try to delete and recreate the POC job I
> > > > > > created at that
> > > > > > link to see if we can get it working?
> > > > > >
> > > > > >
> > > > > I see another email thread on the list saying that we are
> > > > > part
> > > > > of said
> > > > > "related projects". We are expected to move to
> > > > > http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
> > > > > already
> > > > > being removed/migrated from the "H#" pool.
> > > > >
> > > > > Also this:
> > > > >
> > > > > There are over 400 plugins on the current builds.apache.org -
> > > > > most of
> > > which
> > > > > we don't need any more, or are replaced with different
> > > > > plugins
> > > > > on the
> > > new
> > > > > system. I expect there may be some plugins we still need to
> > > > > install to
> > > get
> > > > > you going again, which is why it is vitally important that
> > > > > you
> > > > > start
> > > > > testing and migrating your jobs over *now*. You should all
> > > > > have
> > > > > auth.
> > > > >
> > > > > Any questions, feel free to email the
> > > > > hadoop-migrations@apache.org
> > > list if
> > > > > you are one of the projects listed below. The rest of you,
> > > > > not
> > > > > listed, a
> > > > > similar email to this one will be posted for you shortly on
> > > > > builds@a.o.
> > > > >
> > > > >
> > > > > full details
> > > > >
> > > https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > > > > Patrick
> > > > >
> > > > >
> > > > >
> > > > > > Patrick
> > > > > >
> > > > > >
> > > > > > > ---------- Forwarded message ---------
> > > > > > > Da: Gavin McDonald <gm...@apache.org>
> > > > > > > Date: Gio 16 Lug 2020, 18:33
> > > > > > > Subject: [IMPORTANT] - Migration to ci-builds.a.o
> > > > > > > To: builds <bu...@apache.org>
> > > > > > >
> > > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > This NOTICE is for everyone on builds.apache.org. We are
> > > > > > > migrating
> > > to a
> > > > > > > new
> > > > > > > Cloudbees based Client Master called
> > > > > > > https://ci-builds.apache.org.
> > > The
> > > > > > > migrations of all jobs needs to be done before the switch
> > > > > > > off date of
> > > > > > > 15th
> > > > > > > August 2020, so you have a maximum of 4 weeks.
> > > > > > >
> > > > > > > There is no tool or automated way of migrating your jobs,
> > > > > > > the
> > > > > > > differences in the platforms, the plugins and the setup
> > > > > > > makes it
> > > > > > > impossible
> > > > > > > to do in a safe way. So, you all need to start creating
> > > > > > > new
> > > > > > > jobs on
> > > > > > > ci-infra.a.o and then , when you are happy, turn off your
> > > > > > > old builds
> > > on
> > > > > > > builds.a.o.
> > > > > > >
> > > > > > > There are currently 4 agents over there ready to take
> > > > > > > jobs,
> > > > > > > plus a
> > > > > > > floating
> > > > > > > agent which is shared amongst many masters (more to
> > > > > > > come).
> > > > > > > I will
> > > migrate
> > > > > > > away 2 more agents from builds.a.o to ci-builds.a.o every
> > > > > > > few days,
> > > and
> > > > > > > will keep an eye of load across both and adjust
> > > > > > > accordingly.
> > > > > > >
> > > > > > > If needed, create a ticket on INFRA jira for any issues
> > > > > > > that crop up,
> > > or
> > > > > > > email here on builds@a.o. there may be one or two plugins
> > > > > > > we need to
> > > > > > > install/tweak etc.
> > > > > > >
> > > > > > > We will be not using 'Views' at the top level, but rather
> > > > > > > we will take
> > > > > > > advantage of 'Folders Plus' - each project will get its
> > > > > > > own
> > > > > > > Folder and
> > > > > > > have
> > > > > > > authorisation access to create/edit jobs etc within that
> > > > > > > folder. I
> > > will
> > > > > > > create these folders as you ask for them to start with.
> > > > > > > This setup
> > > allows
> > > > > > > for credentials isolation amongst other benefits,
> > > > > > > including
> > > > > > > but not
> > > > > > > limited
> > > > > > > to exclusive agents (Controlled Agents) for your own use
> > > > > > > ,
> > > > > > > should you
> > > > > > > have
> > > > > > > any project targeted donations of agents.
> > > > > > >
> > > > > > > As with other aspects of the ASF, projects can choose to
> > > > > > > just enable
> > > all
> > > > > > > committers access to their folder, just ask.
> > > > > > >
> > > > > > > We will re-use builds.apache.org as a CNAME to ci-
> > > > > > > builds.a.o but will
> > > > > > > not
> > > > > > > be setting up any forwarding rules or anything like that.
> > > > > > >
> > > > > > > So, please, get started *now *on this so you can be sure
> > > > > > > we
> > > > > > > are all
> > > > > > > completed before the final cutoff date of 15th August
> > > > > > > 2020.
> > > > > > >
> > > > > > > Any questions - I expect a few (dozen :) ) - ask away
> > > > > > > and/or file
> > > INFRA
> > > > > > > tickets.
> > > > > > >
> > > > > > > Hadoop and related projects have their own migration path
> > > > > > > to follow,
> > > same
> > > > > > > cut off date, Cassandra, Beam, CouchDB have already
> > > > > > > migrated and are
> > > > > > > doing
> > > > > > > very well in their new homes.
> > > > > > >
> > > > > > > Lets get going ...
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > *Gavin McDonald*
> > > > > > > Systems Administrator
> > > > > > > ASF Infrastructure Team
> > > > > > >
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi,
I created a few new jobs based on your suggestions. Before going
forward please take a look at them.
I created Jdk LTS jobs (8 and 11) for all active branches: 3.5, 3.6 and
master. Also added the Maven option surefire.forkcount=4 to all jobs to
limit the number of concurrent threads in unit tests.
Do you think we also need non-LTS Java build jobs too?
https://ci-hadoop.apache.org/view/ZooKeeper/
Andor
On Sun, 2020-07-19 at 07:50 +0200, Enrico Olivelli wrote:
> Il Sab 18 Lug 2020, 21:20 Andor Molnar <an...@apache.org> ha scritto:
>
> > Hi Pat,
> >
> > I have admin rights in the new system too and probably can work on
> > this on
> > Monday.
> > What’s “matrix” config? Shouldn’t we just replicate the same jobs
> > on the
> > new instance?
> >
>
> This is my list of important jobs:
> - maven builds of master against every supported jdk +EA (for JDK
> outreach
> quality program)
> - maven build of master on windows on jdk8
> - maven build of branch-3.5 and branch-3.6
> - ant build of branch-3.5 on jdk8
> - maven precommit job
>
> Enrico
>
>
> > Andor
> >
> >
> >
> > > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> > >
> > > I updated the job I linked earlier based on what's the latest on
> > > the
> > legacy
> > > jenkins. It ran successfully
> > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> > >
> > > I didn't replicate every config setting - main gap is the
> > > spotbugs
> > > post-build, which seems to be missing from the new jenkins
> > > plugins.
> > >
> > > That's just maven master though. Not sure about the rest. Can we
> > > do more
> > of
> > > a "matrix" config in the new system vs cloning all the time?
> > >
> > > Patrick
> > >
> > > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org>
> > > wrote:
> > >
> > > > On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org>
> > > > wrote:
> > > >
> > > > >
> > > > > On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <
> > > > > eolivelli@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > FYI
> > > > > >
> > > > >
> > > > > Fun. I do notice it says "Hadoop and related projects have
> > > > > their own
> > > > > migration path to follow" - any insight on that? We are or
> > > > > are not
> > lumped
> > > > > in? I would assume we are?
> > > > >
> > > > > This (eventual migration) came up a while back on the Hadoop
> > > > > PMC and I
> > > > > volunteered to try for us (ZK). I was never able to get it to
> > > > > work, I
> > > > > provided feedback to infra but they never got back, as such
> > > > > we have a
> > > > > project here that's not working with some basic dependencies
> > > > > missing:
> > > > > https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> > > > >
> > > > > That said, we can try again. Can we verify where ZK is
> > > > > supposed to
> > land?
> > > > > Perhaps we can try to delete and recreate the POC job I
> > > > > created at that
> > > > > link to see if we can get it working?
> > > > >
> > > > >
> > > > I see another email thread on the list saying that we are part
> > > > of said
> > > > "related projects". We are expected to move to
> > > > http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are
> > > > already
> > > > being removed/migrated from the "H#" pool.
> > > >
> > > > Also this:
> > > >
> > > > There are over 400 plugins on the current builds.apache.org -
> > > > most of
> > which
> > > > we don't need any more, or are replaced with different plugins
> > > > on the
> > new
> > > > system. I expect there may be some plugins we still need to
> > > > install to
> > get
> > > > you going again, which is why it is vitally important that you
> > > > start
> > > > testing and migrating your jobs over *now*. You should all have
> > > > auth.
> > > >
> > > > Any questions, feel free to email the
> > > > hadoop-migrations@apache.org
> > list if
> > > > you are one of the projects listed below. The rest of you, not
> > > > listed, a
> > > > similar email to this one will be posted for you shortly on
> > > > builds@a.o.
> > > >
> > > >
> > > > full details
> > > >
> > https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> > > > Patrick
> > > >
> > > >
> > > >
> > > > > Patrick
> > > > >
> > > > >
> > > > > > ---------- Forwarded message ---------
> > > > > > Da: Gavin McDonald <gm...@apache.org>
> > > > > > Date: Gio 16 Lug 2020, 18:33
> > > > > > Subject: [IMPORTANT] - Migration to ci-builds.a.o
> > > > > > To: builds <bu...@apache.org>
> > > > > >
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > This NOTICE is for everyone on builds.apache.org. We are
> > > > > > migrating
> > to a
> > > > > > new
> > > > > > Cloudbees based Client Master called
> > > > > > https://ci-builds.apache.org.
> > The
> > > > > > migrations of all jobs needs to be done before the switch
> > > > > > off date of
> > > > > > 15th
> > > > > > August 2020, so you have a maximum of 4 weeks.
> > > > > >
> > > > > > There is no tool or automated way of migrating your jobs,
> > > > > > the
> > > > > > differences in the platforms, the plugins and the setup
> > > > > > makes it
> > > > > > impossible
> > > > > > to do in a safe way. So, you all need to start creating new
> > > > > > jobs on
> > > > > > ci-infra.a.o and then , when you are happy, turn off your
> > > > > > old builds
> > on
> > > > > > builds.a.o.
> > > > > >
> > > > > > There are currently 4 agents over there ready to take jobs,
> > > > > > plus a
> > > > > > floating
> > > > > > agent which is shared amongst many masters (more to come).
> > > > > > I will
> > migrate
> > > > > > away 2 more agents from builds.a.o to ci-builds.a.o every
> > > > > > few days,
> > and
> > > > > > will keep an eye of load across both and adjust
> > > > > > accordingly.
> > > > > >
> > > > > > If needed, create a ticket on INFRA jira for any issues
> > > > > > that crop up,
> > or
> > > > > > email here on builds@a.o. there may be one or two plugins
> > > > > > we need to
> > > > > > install/tweak etc.
> > > > > >
> > > > > > We will be not using 'Views' at the top level, but rather
> > > > > > we will take
> > > > > > advantage of 'Folders Plus' - each project will get its own
> > > > > > Folder and
> > > > > > have
> > > > > > authorisation access to create/edit jobs etc within that
> > > > > > folder. I
> > will
> > > > > > create these folders as you ask for them to start with.
> > > > > > This setup
> > allows
> > > > > > for credentials isolation amongst other benefits, including
> > > > > > but not
> > > > > > limited
> > > > > > to exclusive agents (Controlled Agents) for your own use ,
> > > > > > should you
> > > > > > have
> > > > > > any project targeted donations of agents.
> > > > > >
> > > > > > As with other aspects of the ASF, projects can choose to
> > > > > > just enable
> > all
> > > > > > committers access to their folder, just ask.
> > > > > >
> > > > > > We will re-use builds.apache.org as a CNAME to ci-
> > > > > > builds.a.o but will
> > > > > > not
> > > > > > be setting up any forwarding rules or anything like that.
> > > > > >
> > > > > > So, please, get started *now *on this so you can be sure we
> > > > > > are all
> > > > > > completed before the final cutoff date of 15th August 2020.
> > > > > >
> > > > > > Any questions - I expect a few (dozen :) ) - ask away
> > > > > > and/or file
> > INFRA
> > > > > > tickets.
> > > > > >
> > > > > > Hadoop and related projects have their own migration path
> > > > > > to follow,
> > same
> > > > > > cut off date, Cassandra, Beam, CouchDB have already
> > > > > > migrated and are
> > > > > > doing
> > > > > > very well in their new homes.
> > > > > >
> > > > > > Lets get going ...
> > > > > >
> > > > > > --
> > > > > >
> > > > > > *Gavin McDonald*
> > > > > > Systems Administrator
> > > > > > ASF Infrastructure Team
> > > > > >
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Enrico Olivelli <eo...@gmail.com>.
Il Sab 18 Lug 2020, 21:20 Andor Molnar <an...@apache.org> ha scritto:
> Hi Pat,
>
> I have admin rights in the new system too and probably can work on this on
> Monday.
> What’s “matrix” config? Shouldn’t we just replicate the same jobs on the
> new instance?
>
This is my list of important jobs:
- maven builds of master against every supported jdk +EA (for JDK outreach
quality program)
- maven build of master on windows on jdk8
- maven build of branch-3.5 and branch-3.6
- ant build of branch-3.5 on jdk8
- maven precommit job
Enrico
> Andor
>
>
>
> > On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
> >
> > I updated the job I linked earlier based on what's the latest on the
> legacy
> > jenkins. It ran successfully
> > https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
> >
> > I didn't replicate every config setting - main gap is the spotbugs
> > post-build, which seems to be missing from the new jenkins plugins.
> >
> > That's just maven master though. Not sure about the rest. Can we do more
> of
> > a "matrix" config in the new system vs cloning all the time?
> >
> > Patrick
> >
> > On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org> wrote:
> >
> >>
> >> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org> wrote:
> >>
> >>>
> >>>
> >>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
> >>> wrote:
> >>>
> >>>> FYI
> >>>>
> >>>
> >>> Fun. I do notice it says "Hadoop and related projects have their own
> >>> migration path to follow" - any insight on that? We are or are not
> lumped
> >>> in? I would assume we are?
> >>>
> >>> This (eventual migration) came up a while back on the Hadoop PMC and I
> >>> volunteered to try for us (ZK). I was never able to get it to work, I
> >>> provided feedback to infra but they never got back, as such we have a
> >>> project here that's not working with some basic dependencies missing:
> >>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
> >>>
> >>> That said, we can try again. Can we verify where ZK is supposed to
> land?
> >>> Perhaps we can try to delete and recreate the POC job I created at that
> >>> link to see if we can get it working?
> >>>
> >>>
> >> I see another email thread on the list saying that we are part of said
> >> "related projects". We are expected to move to
> >> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are already
> >> being removed/migrated from the "H#" pool.
> >>
> >> Also this:
> >>
> >> There are over 400 plugins on the current builds.apache.org - most of
> which
> >> we don't need any more, or are replaced with different plugins on the
> new
> >> system. I expect there may be some plugins we still need to install to
> get
> >> you going again, which is why it is vitally important that you start
> >> testing and migrating your jobs over *now*. You should all have auth.
> >>
> >> Any questions, feel free to email the hadoop-migrations@apache.org
> list if
> >> you are one of the projects listed below. The rest of you, not listed, a
> >> similar email to this one will be posted for you shortly on builds@a.o.
> >>
> >>
> >> full details
> >>
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
> >>
> >> Patrick
> >>
> >>
> >>
> >>> Patrick
> >>>
> >>>
> >>>>
> >>>> ---------- Forwarded message ---------
> >>>> Da: Gavin McDonald <gm...@apache.org>
> >>>> Date: Gio 16 Lug 2020, 18:33
> >>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> >>>> To: builds <bu...@apache.org>
> >>>>
> >>>>
> >>>> Hi All,
> >>>>
> >>>> This NOTICE is for everyone on builds.apache.org. We are migrating
> to a
> >>>> new
> >>>> Cloudbees based Client Master called https://ci-builds.apache.org.
> The
> >>>> migrations of all jobs needs to be done before the switch off date of
> >>>> 15th
> >>>> August 2020, so you have a maximum of 4 weeks.
> >>>>
> >>>> There is no tool or automated way of migrating your jobs, the
> >>>> differences in the platforms, the plugins and the setup makes it
> >>>> impossible
> >>>> to do in a safe way. So, you all need to start creating new jobs on
> >>>> ci-infra.a.o and then , when you are happy, turn off your old builds
> on
> >>>> builds.a.o.
> >>>>
> >>>> There are currently 4 agents over there ready to take jobs, plus a
> >>>> floating
> >>>> agent which is shared amongst many masters (more to come). I will
> migrate
> >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days,
> and
> >>>> will keep an eye of load across both and adjust accordingly.
> >>>>
> >>>> If needed, create a ticket on INFRA jira for any issues that crop up,
> or
> >>>> email here on builds@a.o. there may be one or two plugins we need to
> >>>> install/tweak etc.
> >>>>
> >>>> We will be not using 'Views' at the top level, but rather we will take
> >>>> advantage of 'Folders Plus' - each project will get its own Folder and
> >>>> have
> >>>> authorisation access to create/edit jobs etc within that folder. I
> will
> >>>> create these folders as you ask for them to start with. This setup
> allows
> >>>> for credentials isolation amongst other benefits, including but not
> >>>> limited
> >>>> to exclusive agents (Controlled Agents) for your own use , should you
> >>>> have
> >>>> any project targeted donations of agents.
> >>>>
> >>>> As with other aspects of the ASF, projects can choose to just enable
> all
> >>>> committers access to their folder, just ask.
> >>>>
> >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> >>>> not
> >>>> be setting up any forwarding rules or anything like that.
> >>>>
> >>>> So, please, get started *now *on this so you can be sure we are all
> >>>> completed before the final cutoff date of 15th August 2020.
> >>>>
> >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file
> INFRA
> >>>> tickets.
> >>>>
> >>>> Hadoop and related projects have their own migration path to follow,
> same
> >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
> >>>> doing
> >>>> very well in their new homes.
> >>>>
> >>>> Lets get going ...
> >>>>
> >>>> --
> >>>>
> >>>> *Gavin McDonald*
> >>>> Systems Administrator
> >>>> ASF Infrastructure Team
> >>>>
> >>>
>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andor Molnar <an...@apache.org>.
Hi Pat,
I have admin rights in the new system too and probably can work on this on Monday.
What’s “matrix” config? Shouldn’t we just replicate the same jobs on the new instance?
Andor
> On 2020. Jul 17., at 2:51, Patrick Hunt <ph...@apache.org> wrote:
>
> I updated the job I linked earlier based on what's the latest on the legacy
> jenkins. It ran successfully
> https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
>
> I didn't replicate every config setting - main gap is the spotbugs
> post-build, which seems to be missing from the new jenkins plugins.
>
> That's just maven master though. Not sure about the rest. Can we do more of
> a "matrix" config in the new system vs cloning all the time?
>
> Patrick
>
> On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org> wrote:
>
>>
>> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org> wrote:
>>
>>>
>>>
>>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
>>> wrote:
>>>
>>>> FYI
>>>>
>>>
>>> Fun. I do notice it says "Hadoop and related projects have their own
>>> migration path to follow" - any insight on that? We are or are not lumped
>>> in? I would assume we are?
>>>
>>> This (eventual migration) came up a while back on the Hadoop PMC and I
>>> volunteered to try for us (ZK). I was never able to get it to work, I
>>> provided feedback to infra but they never got back, as such we have a
>>> project here that's not working with some basic dependencies missing:
>>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>>
>>> That said, we can try again. Can we verify where ZK is supposed to land?
>>> Perhaps we can try to delete and recreate the POC job I created at that
>>> link to see if we can get it working?
>>>
>>>
>> I see another email thread on the list saying that we are part of said
>> "related projects". We are expected to move to
>> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are already
>> being removed/migrated from the "H#" pool.
>>
>> Also this:
>>
>> There are over 400 plugins on the current builds.apache.org - most of which
>> we don't need any more, or are replaced with different plugins on the new
>> system. I expect there may be some plugins we still need to install to get
>> you going again, which is why it is vitally important that you start
>> testing and migrating your jobs over *now*. You should all have auth.
>>
>> Any questions, feel free to email the hadoop-migrations@apache.org list if
>> you are one of the projects listed below. The rest of you, not listed, a
>> similar email to this one will be posted for you shortly on builds@a.o.
>>
>>
>> full details
>> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>>
>> Patrick
>>
>>
>>
>>> Patrick
>>>
>>>
>>>>
>>>> ---------- Forwarded message ---------
>>>> Da: Gavin McDonald <gm...@apache.org>
>>>> Date: Gio 16 Lug 2020, 18:33
>>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>>>> To: builds <bu...@apache.org>
>>>>
>>>>
>>>> Hi All,
>>>>
>>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
>>>> new
>>>> Cloudbees based Client Master called https://ci-builds.apache.org. The
>>>> migrations of all jobs needs to be done before the switch off date of
>>>> 15th
>>>> August 2020, so you have a maximum of 4 weeks.
>>>>
>>>> There is no tool or automated way of migrating your jobs, the
>>>> differences in the platforms, the plugins and the setup makes it
>>>> impossible
>>>> to do in a safe way. So, you all need to start creating new jobs on
>>>> ci-infra.a.o and then , when you are happy, turn off your old builds on
>>>> builds.a.o.
>>>>
>>>> There are currently 4 agents over there ready to take jobs, plus a
>>>> floating
>>>> agent which is shared amongst many masters (more to come). I will migrate
>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
>>>> will keep an eye of load across both and adjust accordingly.
>>>>
>>>> If needed, create a ticket on INFRA jira for any issues that crop up, or
>>>> email here on builds@a.o. there may be one or two plugins we need to
>>>> install/tweak etc.
>>>>
>>>> We will be not using 'Views' at the top level, but rather we will take
>>>> advantage of 'Folders Plus' - each project will get its own Folder and
>>>> have
>>>> authorisation access to create/edit jobs etc within that folder. I will
>>>> create these folders as you ask for them to start with. This setup allows
>>>> for credentials isolation amongst other benefits, including but not
>>>> limited
>>>> to exclusive agents (Controlled Agents) for your own use , should you
>>>> have
>>>> any project targeted donations of agents.
>>>>
>>>> As with other aspects of the ASF, projects can choose to just enable all
>>>> committers access to their folder, just ask.
>>>>
>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
>>>> not
>>>> be setting up any forwarding rules or anything like that.
>>>>
>>>> So, please, get started *now *on this so you can be sure we are all
>>>> completed before the final cutoff date of 15th August 2020.
>>>>
>>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
>>>> tickets.
>>>>
>>>> Hadoop and related projects have their own migration path to follow, same
>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
>>>> doing
>>>> very well in their new homes.
>>>>
>>>> Lets get going ...
>>>>
>>>> --
>>>>
>>>> *Gavin McDonald*
>>>> Systems Administrator
>>>> ASF Infrastructure Team
>>>>
>>>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
I updated the job I linked earlier based on what's the latest on the legacy
jenkins. It ran successfully
https://ci-hadoop.apache.org/job/zookeeper-master-maven/215/
I didn't replicate every config setting - main gap is the spotbugs
post-build, which seems to be missing from the new jenkins plugins.
That's just maven master though. Not sure about the rest. Can we do more of
a "matrix" config in the new system vs cloning all the time?
Patrick
On Thu, Jul 16, 2020 at 3:31 PM Patrick Hunt <ph...@apache.org> wrote:
>
> On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org> wrote:
>
>>
>>
>> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
>> wrote:
>>
>>> FYI
>>>
>>
>> Fun. I do notice it says "Hadoop and related projects have their own
>> migration path to follow" - any insight on that? We are or are not lumped
>> in? I would assume we are?
>>
>> This (eventual migration) came up a while back on the Hadoop PMC and I
>> volunteered to try for us (ZK). I was never able to get it to work, I
>> provided feedback to infra but they never got back, as such we have a
>> project here that's not working with some basic dependencies missing:
>> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>>
>> That said, we can try again. Can we verify where ZK is supposed to land?
>> Perhaps we can try to delete and recreate the POC job I created at that
>> link to see if we can get it working?
>>
>>
> I see another email thread on the list saying that we are part of said
> "related projects". We are expected to move to
> http://ci-hadoop.apache.org/ within 4 weeks. Seems nodes are already
> being removed/migrated from the "H#" pool.
>
> Also this:
>
> There are over 400 plugins on the current builds.apache.org - most of which
> we don't need any more, or are replaced with different plugins on the new
> system. I expect there may be some plugins we still need to install to get
> you going again, which is why it is vitally important that you start
> testing and migrating your jobs over *now*. You should all have auth.
>
> Any questions, feel free to email the hadoop-migrations@apache.org list if
> you are one of the projects listed below. The rest of you, not listed, a
> similar email to this one will be posted for you shortly on builds@a.o.
>
>
> full details
> https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
>
> Patrick
>
>
>
>> Patrick
>>
>>
>>>
>>> ---------- Forwarded message ---------
>>> Da: Gavin McDonald <gm...@apache.org>
>>> Date: Gio 16 Lug 2020, 18:33
>>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>>> To: builds <bu...@apache.org>
>>>
>>>
>>> Hi All,
>>>
>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
>>> new
>>> Cloudbees based Client Master called https://ci-builds.apache.org. The
>>> migrations of all jobs needs to be done before the switch off date of
>>> 15th
>>> August 2020, so you have a maximum of 4 weeks.
>>>
>>> There is no tool or automated way of migrating your jobs, the
>>> differences in the platforms, the plugins and the setup makes it
>>> impossible
>>> to do in a safe way. So, you all need to start creating new jobs on
>>> ci-infra.a.o and then , when you are happy, turn off your old builds on
>>> builds.a.o.
>>>
>>> There are currently 4 agents over there ready to take jobs, plus a
>>> floating
>>> agent which is shared amongst many masters (more to come). I will migrate
>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
>>> will keep an eye of load across both and adjust accordingly.
>>>
>>> If needed, create a ticket on INFRA jira for any issues that crop up, or
>>> email here on builds@a.o. there may be one or two plugins we need to
>>> install/tweak etc.
>>>
>>> We will be not using 'Views' at the top level, but rather we will take
>>> advantage of 'Folders Plus' - each project will get its own Folder and
>>> have
>>> authorisation access to create/edit jobs etc within that folder. I will
>>> create these folders as you ask for them to start with. This setup allows
>>> for credentials isolation amongst other benefits, including but not
>>> limited
>>> to exclusive agents (Controlled Agents) for your own use , should you
>>> have
>>> any project targeted donations of agents.
>>>
>>> As with other aspects of the ASF, projects can choose to just enable all
>>> committers access to their folder, just ask.
>>>
>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
>>> not
>>> be setting up any forwarding rules or anything like that.
>>>
>>> So, please, get started *now *on this so you can be sure we are all
>>> completed before the final cutoff date of 15th August 2020.
>>>
>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
>>> tickets.
>>>
>>> Hadoop and related projects have their own migration path to follow, same
>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
>>> doing
>>> very well in their new homes.
>>>
>>> Lets get going ...
>>>
>>> --
>>>
>>> *Gavin McDonald*
>>> Systems Administrator
>>> ASF Infrastructure Team
>>>
>>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
On Thu, Jul 16, 2020 at 3:22 PM Patrick Hunt <ph...@apache.org> wrote:
>
>
> On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
>> FYI
>>
>
> Fun. I do notice it says "Hadoop and related projects have their own
> migration path to follow" - any insight on that? We are or are not lumped
> in? I would assume we are?
>
> This (eventual migration) came up a while back on the Hadoop PMC and I
> volunteered to try for us (ZK). I was never able to get it to work, I
> provided feedback to infra but they never got back, as such we have a
> project here that's not working with some basic dependencies missing:
> https://ci-hadoop.apache.org/job/zookeeper-master-maven/
>
> That said, we can try again. Can we verify where ZK is supposed to land?
> Perhaps we can try to delete and recreate the POC job I created at that
> link to see if we can get it working?
>
>
I see another email thread on the list saying that we are part of said
"related projects". We are expected to move to http://ci-hadoop.apache.org/
within 4 weeks. Seems nodes are already being removed/migrated from the
"H#" pool.
Also this:
There are over 400 plugins on the current builds.apache.org - most of which
we don't need any more, or are replaced with different plugins on the new
system. I expect there may be some plugins we still need to install to get
you going again, which is why it is vitally important that you start
testing and migrating your jobs over *now*. You should all have auth.
Any questions, feel free to email the hadoop-migrations@apache.org list if
you are one of the projects listed below. The rest of you, not listed, a
similar email to this one will be posted for you shortly on builds@a.o.
full details
https://lists.apache.org/thread.html/r21c9d40cdbf5461143dd7eb4ff48a200c2fd20c50e946884f61318fd%40%3Cbuilds.apache.org%3E
Patrick
> Patrick
>
>
>>
>> ---------- Forwarded message ---------
>> Da: Gavin McDonald <gm...@apache.org>
>> Date: Gio 16 Lug 2020, 18:33
>> Subject: [IMPORTANT] - Migration to ci-builds.a.o
>> To: builds <bu...@apache.org>
>>
>>
>> Hi All,
>>
>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
>> new
>> Cloudbees based Client Master called https://ci-builds.apache.org. The
>> migrations of all jobs needs to be done before the switch off date of 15th
>> August 2020, so you have a maximum of 4 weeks.
>>
>> There is no tool or automated way of migrating your jobs, the
>> differences in the platforms, the plugins and the setup makes it
>> impossible
>> to do in a safe way. So, you all need to start creating new jobs on
>> ci-infra.a.o and then , when you are happy, turn off your old builds on
>> builds.a.o.
>>
>> There are currently 4 agents over there ready to take jobs, plus a
>> floating
>> agent which is shared amongst many masters (more to come). I will migrate
>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
>> will keep an eye of load across both and adjust accordingly.
>>
>> If needed, create a ticket on INFRA jira for any issues that crop up, or
>> email here on builds@a.o. there may be one or two plugins we need to
>> install/tweak etc.
>>
>> We will be not using 'Views' at the top level, but rather we will take
>> advantage of 'Folders Plus' - each project will get its own Folder and
>> have
>> authorisation access to create/edit jobs etc within that folder. I will
>> create these folders as you ask for them to start with. This setup allows
>> for credentials isolation amongst other benefits, including but not
>> limited
>> to exclusive agents (Controlled Agents) for your own use , should you have
>> any project targeted donations of agents.
>>
>> As with other aspects of the ASF, projects can choose to just enable all
>> committers access to their folder, just ask.
>>
>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
>> be setting up any forwarding rules or anything like that.
>>
>> So, please, get started *now *on this so you can be sure we are all
>> completed before the final cutoff date of 15th August 2020.
>>
>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
>> tickets.
>>
>> Hadoop and related projects have their own migration path to follow, same
>> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
>> very well in their new homes.
>>
>> Lets get going ...
>>
>> --
>>
>> *Gavin McDonald*
>> Systems Administrator
>> ASF Infrastructure Team
>>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Patrick Hunt <ph...@apache.org>.
On Thu, Jul 16, 2020 at 12:56 PM Enrico Olivelli <eo...@gmail.com>
wrote:
> FYI
>
Fun. I do notice it says "Hadoop and related projects have their own
migration path to follow" - any insight on that? We are or are not lumped
in? I would assume we are?
This (eventual migration) came up a while back on the Hadoop PMC and I
volunteered to try for us (ZK). I was never able to get it to work, I
provided feedback to infra but they never got back, as such we have a
project here that's not working with some basic dependencies missing:
https://ci-hadoop.apache.org/job/zookeeper-master-maven/
That said, we can try again. Can we verify where ZK is supposed to land?
Perhaps we can try to delete and recreate the POC job I created at that
link to see if we can get it working?
Patrick
>
> ---------- Forwarded message ---------
> Da: Gavin McDonald <gm...@apache.org>
> Date: Gio 16 Lug 2020, 18:33
> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> To: builds <bu...@apache.org>
>
>
> Hi All,
>
> This NOTICE is for everyone on builds.apache.org. We are migrating to a
> new
> Cloudbees based Client Master called https://ci-builds.apache.org. The
> migrations of all jobs needs to be done before the switch off date of 15th
> August 2020, so you have a maximum of 4 weeks.
>
> There is no tool or automated way of migrating your jobs, the
> differences in the platforms, the plugins and the setup makes it impossible
> to do in a safe way. So, you all need to start creating new jobs on
> ci-infra.a.o and then , when you are happy, turn off your old builds on
> builds.a.o.
>
> There are currently 4 agents over there ready to take jobs, plus a floating
> agent which is shared amongst many masters (more to come). I will migrate
> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> will keep an eye of load across both and adjust accordingly.
>
> If needed, create a ticket on INFRA jira for any issues that crop up, or
> email here on builds@a.o. there may be one or two plugins we need to
> install/tweak etc.
>
> We will be not using 'Views' at the top level, but rather we will take
> advantage of 'Folders Plus' - each project will get its own Folder and have
> authorisation access to create/edit jobs etc within that folder. I will
> create these folders as you ask for them to start with. This setup allows
> for credentials isolation amongst other benefits, including but not limited
> to exclusive agents (Controlled Agents) for your own use , should you have
> any project targeted donations of agents.
>
> As with other aspects of the ASF, projects can choose to just enable all
> committers access to their folder, just ask.
>
> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
> be setting up any forwarding rules or anything like that.
>
> So, please, get started *now *on this so you can be sure we are all
> completed before the final cutoff date of 15th August 2020.
>
> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> tickets.
>
> Hadoop and related projects have their own migration path to follow, same
> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
> very well in their new homes.
>
> Lets get going ...
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>