You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Gézapeti Cseh <ge...@apache.org> on 2020/07/22 16:18:04 UTC
Fwd: [IMPORTANT] - Migration to ci-builds.a.o
Hey,
After a short discussion on private@ (see the initial mail below) we've
decided to move the jobs 6) and 7) over.
I'd like to ask the Hadoop mailing list so Oozie can join the
https://ci-hadoop.apache.org/ folder and use Hadoop's infrastructure (and
pre-commit stuff) going forward just like we do today.
Please let me know if there are any objections.
Thanks
gp
---------- Forwarded message ---------
From: Gézapeti Cseh <ge...@apache.org>
Date: Thu, Jul 16, 2020 at 8:24 PM
Subject: Fwd: [IMPORTANT] - Migration to ci-builds.a.o
To: <pr...@oozie.apache.org>
Hey Oozie PMC!
I've come across this mail on the builds@ list.
I've found the following jobs on the old servers related to Oozie:
1. oozie-master-testing
<https://builds.apache.org/search/?q=oozie-master-testing>
2. oozie-trunk-w-hadoop-1
<https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-1>
3. oozie-trunk-w-hadoop-2
<https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-2>
4. oozie-trunk-precommit-build
<https://builds.apache.org/search/?q=oozie-trunk-precommit-build>
5. oozie-trunk-find-patches-available
<https://builds.apache.org/search/?q=oozie-trunk-find-patches-available>
6. PreCommit-OOZIE-Build
<https://builds.apache.org/search/?q=PreCommit-OOZIE-Build>
7. Oozie-flakytest-reproduction
<https://builds.apache.org/search/?q=Oozie-flakytest-reproduction>
I think the only one that's worth migrating is 6) as it's kicked off by the
pre-commit <https://builds.apache.org/job/PreCommit-Admin/> admin job not
by 5).
I'll try to do the migration, but I'd be happy if someone could join my
adventures on the oozie Slack channel
<https://the-asf.slack.com/archives/CDG1XSSTW>. I'm open to date
suggestions :)
Please let me know if 6) is sufficient or we have something else to migrate.
thanks
gp
---------- Forwarded message ---------
From: Gavin McDonald <gm...@apache.org>
Date: Thu, Jul 16, 2020 at 6:33 PM
Subject: [IMPORTANT] - Migration to ci-builds.a.o
To: builds <bu...@apache.org>
Hi All,
This NOTICE is for everyone on builds.apache.org. We are migrating to a new
Cloudbees based Client Master called https://ci-builds.apache.org. The
migrations of all jobs needs to be done before the switch off date of 15th
August 2020, so you have a maximum of 4 weeks.
There is no tool or automated way of migrating your jobs, the
differences in the platforms, the plugins and the setup makes it impossible
to do in a safe way. So, you all need to start creating new jobs on
ci-infra.a.o and then , when you are happy, turn off your old builds on
builds.a.o.
There are currently 4 agents over there ready to take jobs, plus a floating
agent which is shared amongst many masters (more to come). I will migrate
away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
will keep an eye of load across both and adjust accordingly.
If needed, create a ticket on INFRA jira for any issues that crop up, or
email here on builds@a.o. there may be one or two plugins we need to
install/tweak etc.
We will be not using 'Views' at the top level, but rather we will take
advantage of 'Folders Plus' - each project will get its own Folder and have
authorisation access to create/edit jobs etc within that folder. I will
create these folders as you ask for them to start with. This setup allows
for credentials isolation amongst other benefits, including but not limited
to exclusive agents (Controlled Agents) for your own use , should you have
any project targeted donations of agents.
As with other aspects of the ASF, projects can choose to just enable all
committers access to their folder, just ask.
We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
be setting up any forwarding rules or anything like that.
So, please, get started *now *on this so you can be sure we are all
completed before the final cutoff date of 15th August 2020.
Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
tickets.
Hadoop and related projects have their own migration path to follow, same
cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
very well in their new homes.
Lets get going ...
--
*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Gézapeti Cseh <ge...@apache.org>.
Status report:
After asking for permissions to all PMC members in INFRA-20705
<https://issues.apache.org/jira/browse/INFRA-20705> I've created the new job
<https://ci-hadoop.apache.org/job/PreCommit-OOZIE-Build/>.
The first run
<https://ci-hadoop.apache.org/job/PreCommit-OOZIE-Build/1/console> has been
triggered by the admin job and it failed successfully and the Jira ticket
under test <https://issues.apache.org/jira/browse/OOZIE-3608> was updated
as well.
I think we can call it a day for the Jenkins migration.
Thanks for the input and please file a ticket or call out if you encounter
any issues regarding the new pre-commit job.
gp
On Wed, Jul 22, 2020 at 6:18 PM Gézapeti Cseh <ge...@apache.org> wrote:
> Hey,
>
> After a short discussion on private@ (see the initial mail below) we've
> decided to move the jobs 6) and 7) over.
>
> I'd like to ask the Hadoop mailing list so Oozie can join the
> https://ci-hadoop.apache.org/ folder and use Hadoop's infrastructure (and
> pre-commit stuff) going forward just like we do today.
> Please let me know if there are any objections.
> Thanks
> gp
> ---------- Forwarded message ---------
> From: Gézapeti Cseh <ge...@apache.org>
> Date: Thu, Jul 16, 2020 at 8:24 PM
> Subject: Fwd: [IMPORTANT] - Migration to ci-builds.a.o
> To: <pr...@oozie.apache.org>
>
>
> Hey Oozie PMC!
>
> I've come across this mail on the builds@ list.
> I've found the following jobs on the old servers related to Oozie:
>
> 1. oozie-master-testing
> <https://builds.apache.org/search/?q=oozie-master-testing>
> 2. oozie-trunk-w-hadoop-1
> <https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-1>
> 3. oozie-trunk-w-hadoop-2
> <https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-2>
> 4. oozie-trunk-precommit-build
> <https://builds.apache.org/search/?q=oozie-trunk-precommit-build>
> 5. oozie-trunk-find-patches-available
> <https://builds.apache.org/search/?q=oozie-trunk-find-patches-available
> >
> 6. PreCommit-OOZIE-Build
> <https://builds.apache.org/search/?q=PreCommit-OOZIE-Build>
> 7. Oozie-flakytest-reproduction
> <https://builds.apache.org/search/?q=Oozie-flakytest-reproduction>
>
>
> I think the only one that's worth migrating is 6) as it's kicked off by the
> pre-commit <https://builds.apache.org/job/PreCommit-Admin/> admin job not
> by 5).
>
> I'll try to do the migration, but I'd be happy if someone could join my
> adventures on the oozie Slack channel
> <https://the-asf.slack.com/archives/CDG1XSSTW>. I'm open to date
> suggestions :)
>
> Please let me know if 6) is sufficient or we have something else to
> migrate.
> thanks
> gp
>
> ---------- Forwarded message ---------
> From: Gavin McDonald <gm...@apache.org>
> Date: Thu, Jul 16, 2020 at 6:33 PM
> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> To: builds <bu...@apache.org>
>
>
> Hi All,
>
> This NOTICE is for everyone on builds.apache.org. We are migrating to a
> new
> Cloudbees based Client Master called https://ci-builds.apache.org. The
> migrations of all jobs needs to be done before the switch off date of 15th
> August 2020, so you have a maximum of 4 weeks.
>
> There is no tool or automated way of migrating your jobs, the
> differences in the platforms, the plugins and the setup makes it impossible
> to do in a safe way. So, you all need to start creating new jobs on
> ci-infra.a.o and then , when you are happy, turn off your old builds on
> builds.a.o.
>
> There are currently 4 agents over there ready to take jobs, plus a floating
> agent which is shared amongst many masters (more to come). I will migrate
> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> will keep an eye of load across both and adjust accordingly.
>
> If needed, create a ticket on INFRA jira for any issues that crop up, or
> email here on builds@a.o. there may be one or two plugins we need to
> install/tweak etc.
>
> We will be not using 'Views' at the top level, but rather we will take
> advantage of 'Folders Plus' - each project will get its own Folder and have
> authorisation access to create/edit jobs etc within that folder. I will
> create these folders as you ask for them to start with. This setup allows
> for credentials isolation amongst other benefits, including but not limited
> to exclusive agents (Controlled Agents) for your own use , should you have
> any project targeted donations of agents.
>
> As with other aspects of the ASF, projects can choose to just enable all
> committers access to their folder, just ask.
>
> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
> be setting up any forwarding rules or anything like that.
>
> So, please, get started *now *on this so you can be sure we are all
> completed before the final cutoff date of 15th August 2020.
>
> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> tickets.
>
> Hadoop and related projects have their own migration path to follow, same
> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
> very well in their new homes.
>
> Lets get going ...
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
--
*Peter Cseh* | Software Engineer, Cloudera Search
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------
On Wed, Jul 22, 2020 at 6:18 PM Gézapeti Cseh <ge...@apache.org> wrote:
>
> Hey,
>
> After a short discussion on private@ (see the initial mail below) we've
> decided to move the jobs 6) and 7) over.
>
> I'd like to ask the Hadoop mailing list so Oozie can join the
> https://ci-hadoop.apache.org/ folder and use Hadoop's infrastructure
> (and pre-commit stuff) going forward just like we do today.
> Please let me know if there are any objections.
> Thanks
> gp
> ---------- Forwarded message ---------
> From: Gézapeti Cseh <ge...@apache.org>
> Date: Thu, Jul 16, 2020 at 8:24 PM
> Subject: Fwd: [IMPORTANT] - Migration to ci-builds.a.o
> To: <pr...@oozie.apache.org>
>
>
> Hey Oozie PMC!
>
> I've come across this mail on the builds@ list.
> I've found the following jobs on the old servers related to Oozie:
>
> 1. oozie-master-testing
> <https://builds.apache.org/search/?q=oozie-master-testing>
> 2. oozie-trunk-w-hadoop-1
> <https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-1>
> 3. oozie-trunk-w-hadoop-2
> <https://builds.apache.org/search/?q=oozie-trunk-w-hadoop-2>
> 4. oozie-trunk-precommit-build
> <https://builds.apache.org/search/?q=oozie-trunk-precommit-build>
> 5. oozie-trunk-find-patches-available
> <https://builds.apache.org/search/?q=oozie-trunk-find-patches-available>
> 6. PreCommit-OOZIE-Build
> <https://builds.apache.org/search/?q=PreCommit-OOZIE-Build>
> 7. Oozie-flakytest-reproduction
> <https://builds.apache.org/search/?q=Oozie-flakytest-reproduction>
>
>
> I think the only one that's worth migrating is 6) as it's kicked off by
> the pre-commit <https://builds.apache.org/job/PreCommit-Admin/> admin job
> not by 5).
>
> I'll try to do the migration, but I'd be happy if someone could join my
> adventures on the oozie Slack channel
> <https://the-asf.slack.com/archives/CDG1XSSTW>. I'm open to date
> suggestions :)
>
> Please let me know if 6) is sufficient or we have something else to
> migrate.
> thanks
> gp
>
> ---------- Forwarded message ---------
> From: Gavin McDonald <gm...@apache.org>
> Date: Thu, Jul 16, 2020 at 6:33 PM
> Subject: [IMPORTANT] - Migration to ci-builds.a.o
> To: builds <bu...@apache.org>
>
>
> Hi All,
>
> This NOTICE is for everyone on builds.apache.org. We are migrating to a
> new
> Cloudbees based Client Master called https://ci-builds.apache.org. The
> migrations of all jobs needs to be done before the switch off date of 15th
> August 2020, so you have a maximum of 4 weeks.
>
> There is no tool or automated way of migrating your jobs, the
> differences in the platforms, the plugins and the setup makes it impossible
> to do in a safe way. So, you all need to start creating new jobs on
> ci-infra.a.o and then , when you are happy, turn off your old builds on
> builds.a.o.
>
> There are currently 4 agents over there ready to take jobs, plus a floating
> agent which is shared amongst many masters (more to come). I will migrate
> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> will keep an eye of load across both and adjust accordingly.
>
> If needed, create a ticket on INFRA jira for any issues that crop up, or
> email here on builds@a.o. there may be one or two plugins we need to
> install/tweak etc.
>
> We will be not using 'Views' at the top level, but rather we will take
> advantage of 'Folders Plus' - each project will get its own Folder and have
> authorisation access to create/edit jobs etc within that folder. I will
> create these folders as you ask for them to start with. This setup allows
> for credentials isolation amongst other benefits, including but not limited
> to exclusive agents (Controlled Agents) for your own use , should you have
> any project targeted donations of agents.
>
> As with other aspects of the ASF, projects can choose to just enable all
> committers access to their folder, just ask.
>
> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
> be setting up any forwarding rules or anything like that.
>
> So, please, get started *now *on this so you can be sure we are all
> completed before the final cutoff date of 15th August 2020.
>
> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> tickets.
>
> Hadoop and related projects have their own migration path to follow, same
> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
> very well in their new homes.
>
> Lets get going ...
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>
>
>
>
>