You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2020/07/17 20:21:11 UTC
Fwd: [IMPORTANT] - Migration to ci-builds.a.o
Jenkins is on the move.
I've asked for a Folder on the new system.
I am so glad we have a simple build setup!
Andy
On 16/07/2020 17:33, Gavin McDonald wrote:
> Hi All,
>
> This NOTICE is for everyone on builds.apache.org. We are migrating to a new
> Cloudbees based Client Master called https://ci-builds.apache.org. The
> migrations of all jobs needs to be done before the switch off date of 15th
> August 2020, so you have a maximum of 4 weeks.
>
> There is no tool or automated way of migrating your jobs, the
> differences in the platforms, the plugins and the setup makes it impossible
> to do in a safe way. So, you all need to start creating new jobs on
> ci-infra.a.o and then , when you are happy, turn off your old builds on
> builds.a.o.
>
> There are currently 4 agents over there ready to take jobs, plus a floating
> agent which is shared amongst many masters (more to come). I will migrate
> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> will keep an eye of load across both and adjust accordingly.
>
> If needed, create a ticket on INFRA jira for any issues that crop up, or
> email here on builds@a.o. there may be one or two plugins we need to
> install/tweak etc.
>
> We will be not using 'Views' at the top level, but rather we will take
> advantage of 'Folders Plus' - each project will get its own Folder and have
> authorisation access to create/edit jobs etc within that folder. I will
> create these folders as you ask for them to start with. This setup allows
> for credentials isolation amongst other benefits, including but not limited
> to exclusive agents (Controlled Agents) for your own use , should you have
> any project targeted donations of agents.
>
> As with other aspects of the ASF, projects can choose to just enable all
> committers access to their folder, just ask.
>
> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
> be setting up any forwarding rules or anything like that.
>
> So, please, get started *now *on this so you can be sure we are all
> completed before the final cutoff date of 15th August 2020.
>
> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> tickets.
>
> Hadoop and related projects have their own migration path to follow, same
> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
> very well in their new homes.
>
> Lets get going ...
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Roy Lenferink <rl...@apache.org>.
Pushing to 'asf-site' and 'asf-staging*' branches will be allowed automatically when running on the
git-websites labeled node.
I also just opened a pull request for site building to put the job configuration in version control
instead of being part of the job: https://github.com/apache/jena-site/pull/17
IMO it's a bit nicer to have the job config in git.
On 2020/07/24 12:13:21, Andy Seaborne <an...@apache.org> wrote:
> I've forgotten the details of how security for site update.
>
> Is it that is it runs on the right node (label "git-websites"), it just
> works? i.e. no credential files needed?
>
> ----
>
> The other builds:
>
> They have started adding worker nodes and I've started copying jobs
> over. It seems slower than the main server at the moment and there is a
> massive build queue but they are all waiting on something so Jena jobs
> are starting within a few seconds.
>
> I'm trying to reduce the number of things that are kept (artifacts) and
> setting the various JDK jobs to poll only hourly. All this this can be
> changed, it's just a chance to review setup and play nice.
>
> Jobs on the old server are still there.
>
> Andy
>
>
>
> On 17/07/2020 21:58, Andy Seaborne wrote:
> > Thanks!
> >
> > Fingers-crossed - copying the job across will work. It's not like the
> > site build is unique for Jena.
> >
> > Good timing WRT the recent release as well.
> >
> > Andy
> >
> > On 17/07/2020 21:31, Roy Lenferink wrote:
> >> If any help is needed with the re-creation of the jena-site job let me
> >> know
> >> !
> >>
> >> Roy
> >>
> >> Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne <an...@apache.org>
> >>
> >>> Jenkins is on the move.
> >>>
> >>> I've asked for a Folder on the new system.
> >>>
> >>> I am so glad we have a simple build setup!
> >>>
> >>> Andy
> >>>
> >>>
> >>> On 16/07/2020 17:33, Gavin McDonald wrote:
> >>>> Hi All,
> >>>>
> >>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
> >>> new
> >>>> Cloudbees based Client Master called https://ci-builds.apache.org. The
> >>>> migrations of all jobs needs to be done before the switch off date of
> >>> 15th
> >>>> August 2020, so you have a maximum of 4 weeks.
> >>>>
> >>>> There is no tool or automated way of migrating your jobs, the
> >>>> differences in the platforms, the plugins and the setup makes it
> >>> impossible
> >>>> to do in a safe way. So, you all need to start creating new jobs on
> >>>> ci-infra.a.o and then , when you are happy, turn off your old builds on
> >>>> builds.a.o.
> >>>>
> >>>> There are currently 4 agents over there ready to take jobs, plus a
> >>> floating
> >>>> agent which is shared amongst many masters (more to come). I will
> >>>> migrate
> >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> >>>> will keep an eye of load across both and adjust accordingly.
> >>>>
> >>>> If needed, create a ticket on INFRA jira for any issues that crop
> >>>> up, or
> >>>> email here on builds@a.o. there may be one or two plugins we need to
> >>>> install/tweak etc.
> >>>>
> >>>> We will be not using 'Views' at the top level, but rather we will take
> >>>> advantage of 'Folders Plus' - each project will get its own Folder and
> >>> have
> >>>> authorisation access to create/edit jobs etc within that folder. I will
> >>>> create these folders as you ask for them to start with. This setup
> >>>> allows
> >>>> for credentials isolation amongst other benefits, including but not
> >>> limited
> >>>> to exclusive agents (Controlled Agents) for your own use , should you
> >>> have
> >>>> any project targeted donations of agents.
> >>>>
> >>>> As with other aspects of the ASF, projects can choose to just enable
> >>>> all
> >>>> committers access to their folder, just ask.
> >>>>
> >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> >>> not
> >>>> be setting up any forwarding rules or anything like that.
> >>>>
> >>>> So, please, get started *now *on this so you can be sure we are all
> >>>> completed before the final cutoff date of 15th August 2020.
> >>>>
> >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> >>>> tickets.
> >>>>
> >>>> Hadoop and related projects have their own migration path to follow,
> >>>> same
> >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
> >>> doing
> >>>> very well in their new homes.
> >>>>
> >>>> Lets get going ...
> >>>>
> >>>
> >>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andy Seaborne <an...@apache.org>.
I've forgotten the details of how security for site update.
Is it that is it runs on the right node (label "git-websites"), it just
works? i.e. no credential files needed?
----
The other builds:
They have started adding worker nodes and I've started copying jobs
over. It seems slower than the main server at the moment and there is a
massive build queue but they are all waiting on something so Jena jobs
are starting within a few seconds.
I'm trying to reduce the number of things that are kept (artifacts) and
setting the various JDK jobs to poll only hourly. All this this can be
changed, it's just a chance to review setup and play nice.
Jobs on the old server are still there.
Andy
On 17/07/2020 21:58, Andy Seaborne wrote:
> Thanks!
>
> Fingers-crossed - copying the job across will work. It's not like the
> site build is unique for Jena.
>
> Good timing WRT the recent release as well.
>
> Andy
>
> On 17/07/2020 21:31, Roy Lenferink wrote:
>> If any help is needed with the re-creation of the jena-site job let me
>> know
>> !
>>
>> Roy
>>
>> Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne <an...@apache.org>
>>
>>> Jenkins is on the move.
>>>
>>> I've asked for a Folder on the new system.
>>>
>>> I am so glad we have a simple build setup!
>>>
>>> Andy
>>>
>>>
>>> On 16/07/2020 17:33, Gavin McDonald wrote:
>>>> Hi All,
>>>>
>>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
>>> new
>>>> Cloudbees based Client Master called https://ci-builds.apache.org. The
>>>> migrations of all jobs needs to be done before the switch off date of
>>> 15th
>>>> August 2020, so you have a maximum of 4 weeks.
>>>>
>>>> There is no tool or automated way of migrating your jobs, the
>>>> differences in the platforms, the plugins and the setup makes it
>>> impossible
>>>> to do in a safe way. So, you all need to start creating new jobs on
>>>> ci-infra.a.o and then , when you are happy, turn off your old builds on
>>>> builds.a.o.
>>>>
>>>> There are currently 4 agents over there ready to take jobs, plus a
>>> floating
>>>> agent which is shared amongst many masters (more to come). I will
>>>> migrate
>>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
>>>> will keep an eye of load across both and adjust accordingly.
>>>>
>>>> If needed, create a ticket on INFRA jira for any issues that crop
>>>> up, or
>>>> email here on builds@a.o. there may be one or two plugins we need to
>>>> install/tweak etc.
>>>>
>>>> We will be not using 'Views' at the top level, but rather we will take
>>>> advantage of 'Folders Plus' - each project will get its own Folder and
>>> have
>>>> authorisation access to create/edit jobs etc within that folder. I will
>>>> create these folders as you ask for them to start with. This setup
>>>> allows
>>>> for credentials isolation amongst other benefits, including but not
>>> limited
>>>> to exclusive agents (Controlled Agents) for your own use , should you
>>> have
>>>> any project targeted donations of agents.
>>>>
>>>> As with other aspects of the ASF, projects can choose to just enable
>>>> all
>>>> committers access to their folder, just ask.
>>>>
>>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
>>> not
>>>> be setting up any forwarding rules or anything like that.
>>>>
>>>> So, please, get started *now *on this so you can be sure we are all
>>>> completed before the final cutoff date of 15th August 2020.
>>>>
>>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
>>>> tickets.
>>>>
>>>> Hadoop and related projects have their own migration path to follow,
>>>> same
>>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
>>> doing
>>>> very well in their new homes.
>>>>
>>>> Lets get going ...
>>>>
>>>
>>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Andy Seaborne <an...@apache.org>.
Thanks!
Fingers-crossed - copying the job across will work. It's not like the
site build is unique for Jena.
Good timing WRT the recent release as well.
Andy
On 17/07/2020 21:31, Roy Lenferink wrote:
> If any help is needed with the re-creation of the jena-site job let me know
> !
>
> Roy
>
> Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne <an...@apache.org>
>
>> Jenkins is on the move.
>>
>> I've asked for a Folder on the new system.
>>
>> I am so glad we have a simple build setup!
>>
>> Andy
>>
>>
>> On 16/07/2020 17:33, Gavin McDonald wrote:
>>> Hi All,
>>>
>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a
>> new
>>> Cloudbees based Client Master called https://ci-builds.apache.org. The
>>> migrations of all jobs needs to be done before the switch off date of
>> 15th
>>> August 2020, so you have a maximum of 4 weeks.
>>>
>>> There is no tool or automated way of migrating your jobs, the
>>> differences in the platforms, the plugins and the setup makes it
>> impossible
>>> to do in a safe way. So, you all need to start creating new jobs on
>>> ci-infra.a.o and then , when you are happy, turn off your old builds on
>>> builds.a.o.
>>>
>>> There are currently 4 agents over there ready to take jobs, plus a
>> floating
>>> agent which is shared amongst many masters (more to come). I will migrate
>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
>>> will keep an eye of load across both and adjust accordingly.
>>>
>>> If needed, create a ticket on INFRA jira for any issues that crop up, or
>>> email here on builds@a.o. there may be one or two plugins we need to
>>> install/tweak etc.
>>>
>>> We will be not using 'Views' at the top level, but rather we will take
>>> advantage of 'Folders Plus' - each project will get its own Folder and
>> have
>>> authorisation access to create/edit jobs etc within that folder. I will
>>> create these folders as you ask for them to start with. This setup allows
>>> for credentials isolation amongst other benefits, including but not
>> limited
>>> to exclusive agents (Controlled Agents) for your own use , should you
>> have
>>> any project targeted donations of agents.
>>>
>>> As with other aspects of the ASF, projects can choose to just enable all
>>> committers access to their folder, just ask.
>>>
>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
>> not
>>> be setting up any forwarding rules or anything like that.
>>>
>>> So, please, get started *now *on this so you can be sure we are all
>>> completed before the final cutoff date of 15th August 2020.
>>>
>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
>>> tickets.
>>>
>>> Hadoop and related projects have their own migration path to follow, same
>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are
>> doing
>>> very well in their new homes.
>>>
>>> Lets get going ...
>>>
>>
>
Re: [IMPORTANT] - Migration to ci-builds.a.o
Posted by Roy Lenferink <rl...@apache.org>.
If any help is needed with the re-creation of the jena-site job let me know
!
Roy
Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne <an...@apache.org>
> Jenkins is on the move.
>
> I've asked for a Folder on the new system.
>
> I am so glad we have a simple build setup!
>
> Andy
>
>
> On 16/07/2020 17:33, Gavin McDonald wrote:
> > Hi All,
> >
> > This NOTICE is for everyone on builds.apache.org. We are migrating to a
> new
> > Cloudbees based Client Master called https://ci-builds.apache.org. The
> > migrations of all jobs needs to be done before the switch off date of
> 15th
> > August 2020, so you have a maximum of 4 weeks.
> >
> > There is no tool or automated way of migrating your jobs, the
> > differences in the platforms, the plugins and the setup makes it
> impossible
> > to do in a safe way. So, you all need to start creating new jobs on
> > ci-infra.a.o and then , when you are happy, turn off your old builds on
> > builds.a.o.
> >
> > There are currently 4 agents over there ready to take jobs, plus a
> floating
> > agent which is shared amongst many masters (more to come). I will migrate
> > away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> > will keep an eye of load across both and adjust accordingly.
> >
> > If needed, create a ticket on INFRA jira for any issues that crop up, or
> > email here on builds@a.o. there may be one or two plugins we need to
> > install/tweak etc.
> >
> > We will be not using 'Views' at the top level, but rather we will take
> > advantage of 'Folders Plus' - each project will get its own Folder and
> have
> > authorisation access to create/edit jobs etc within that folder. I will
> > create these folders as you ask for them to start with. This setup allows
> > for credentials isolation amongst other benefits, including but not
> limited
> > to exclusive agents (Controlled Agents) for your own use , should you
> have
> > any project targeted donations of agents.
> >
> > As with other aspects of the ASF, projects can choose to just enable all
> > committers access to their folder, just ask.
> >
> > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> not
> > be setting up any forwarding rules or anything like that.
> >
> > So, please, get started *now *on this so you can be sure we are all
> > completed before the final cutoff date of 15th August 2020.
> >
> > Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> > tickets.
> >
> > Hadoop and related projects have their own migration path to follow, same
> > cut off date, Cassandra, Beam, CouchDB have already migrated and are
> doing
> > very well in their new homes.
> >
> > Lets get going ...
> >
>
Migration to ci-builds.a.o
Posted by Andy Seaborne <an...@apache.org>.
https://ci-builds.apache.org/job/Jena/
On 17/07/2020 21:21, Andy Seaborne wrote:
> Jenkins is on the move.
>
> I've asked for a Folder on the new system.
>
> I am so glad we have a simple build setup!
>
> Andy
I've created the dev-test job. It has github integration enabled so from
changes you can click to go to commits. it also builds from the GH repo.
I simple change to the ASF code repo caused the job to run automatically.
Nothing has been deleted on the old build server but I've disabled
several jobs to reduce the noise.
Basically, looking good.
More jobs to move across as time permits.
The new Jenkins server is still being configured so it is having more
restarts than normal, and is currently under provisioned with job executors.
The Jena website job is the important one but again I hope it's a matter
of check then cut&paste.
(all the other jobs are the same type as the dev-test one with slightly
different settings).
Andy