You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficcontrol.apache.org by Zach Hoffman <za...@zrhoffman.net> on 2020/08/13 16:34:29 UTC
Migrating to ci-builds.apache.org from builds.apache.org
As noted in #dev in the ATC slack yesterday, our Jenkins jobs aren't
getting run, but the issue isn't a simple "Other projects' jobs are
monopolizing the runners".
If you go to builds.apache.org, there is this banner:
> IMPORTANT: This Jenkins server will be turned off on the 15th August,
please migrate your jobs!! See the builds@a.o and/or
hadoop-migrations@infra.a.o mailing list for more details
https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
<builds.apache.org>
Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on the
builds.apache.org mailing list (
https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
<builds.apache.org> ):
> Hi All,
>
> This NOTICE is for everyone on builds.apache.org. We are migrating to a
new
> Cloudbees based Client Master called https://ci-builds.apache.org. The
> migrations of all jobs needs to be done before the switch off date of 15th
> August 2020, so you have a maximum of 4 weeks.
>
> There is no tool or automated way of migrating your jobs, the
> differences in the platforms, the plugins and the setup makes it
impossible
> to do in a safe way. So, you all need to start creating new jobs on
> ci-infra.a.o and then , when you are happy, turn off your old builds on
> builds.a.o.
>
> There are currently 4 agents over there ready to take jobs, plus a
floating
> agent which is shared amongst many masters (more to come). I will migrate
> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> will keep an eye of load across both and adjust accordingly.
>
> If needed, create a ticket on INFRA jira for any issues that crop up, or
> email here on builds@a.o. there may be one or two plugins we need to
> install/tweak etc.
>
> We will be not using 'Views' at the top level, but rather we will take
> advantage of 'Folders Plus' - each project will get its own Folder and
have
> authorisation access to create/edit jobs etc within that folder. I will
> create these folders as you ask for them to start with. This setup allows
> for credentials isolation amongst other benefits, including but not
limited
> to exclusive agents (Controlled Agents) for your own use , should you have
> any project targeted donations of agents.
>
> As with other aspects of the ASF, projects can choose to just enable all
> committers access to their folder, just ask.
>
> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will not
> be setting up any forwarding rules or anything like that.
>
> So, please, get started *now *on this so you can be sure we are all
> completed before the final cutoff date of 15th August 2020.
>
> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> tickets.
>
> Hadoop and related projects have their own migration path to follow, same
> cut off date, Cassandra, Beam, CouchDB have already migrated and are doing
> very well in their new homes.
>
> Lets get going ...
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
2 days until the 15th. Does anyone have access to
https://ci-builds.apache.org/ yet?
On your mark, get set, go,
Zach
Re: Migrating to ci-builds.apache.org from builds.apache.org
Posted by Dave Neuman <ne...@apache.org>.
Does GHA also host the builds for downloads?
One nice thing about Jenkins is that it hosts helper builds that we can
point people in the community to.
On Thu, Aug 13, 2020 at 11:16 AM Jeremy Mitchell <mi...@gmail.com>
wrote:
> I think it's time to move away from apache jenkins and embrace github
> actions (GHA). In addition to providing better visibility on a PR (i.e.
> https://github.com/apache/trafficcontrol/pull/4933/checks), GHAs can be
> created/modified by non-committers whereas apache jenkins cannot.
>
> When this open PR is merged from Brennan:
> https://github.com/apache/trafficcontrol/pull/4948 we will have GHAs that:
>
> - run the weasel license checker
> - check that each TC component builds successfully
> - runs the unit tests agains each component
>
> I "think" this encompasses everything done previously by apache jenkins.
>
> +1 for GHA, -1 for migrating to https://ci-builds.apache.org
>
> Jeremy
>
>
>
> On Thu, Aug 13, 2020 at 10:36 AM Zach Hoffman <za...@zrhoffman.net> wrote:
>
> > Actual link to the builds@apache.org mailing list thread:
> >
> >
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d%40%3Cbuilds.apache.org%3E
> >
> > On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net>
> wrote:
> >
> > > As noted in #dev in the ATC slack yesterday, our Jenkins jobs aren't
> > > getting run, but the issue isn't a simple "Other projects' jobs are
> > > monopolizing the runners".
> > >
> > > If you go to builds.apache.org, there is this banner:
> > > > IMPORTANT: This Jenkins server will be turned off on the 15th August,
> > > please migrate your jobs!! See the builds@a.o and/or
> > > hadoop-migrations@infra.a.o mailing list for more details
> > >
> > >
> > >
> >
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> > > <builds.apache.org>
> > >
> > > Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on the
> > > builds.apache.org mailing list (
> > >
> >
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> > > <builds.apache.org> ):
> > >
> > > > Hi All,
> > > >
> > > > This NOTICE is for everyone on builds.apache.org. We are migrating
> to
> > a
> > > new
> > > > Cloudbees based Client Master called https://ci-builds.apache.org.
> The
> > > > migrations of all jobs needs to be done before the switch off date of
> > > 15th
> > > > August 2020, so you have a maximum of 4 weeks.
> > > >
> > > > There is no tool or automated way of migrating your jobs, the
> > > > differences in the platforms, the plugins and the setup makes it
> > > impossible
> > > > to do in a safe way. So, you all need to start creating new jobs on
> > > > ci-infra.a.o and then , when you are happy, turn off your old builds
> on
> > > > builds.a.o.
> > > >
> > > > There are currently 4 agents over there ready to take jobs, plus a
> > > floating
> > > > agent which is shared amongst many masters (more to come). I will
> > migrate
> > > > away 2 more agents from builds.a.o to ci-builds.a.o every few days,
> and
> > > > will keep an eye of load across both and adjust accordingly.
> > > >
> > > > If needed, create a ticket on INFRA jira for any issues that crop up,
> > or
> > > > email here on builds@a.o. there may be one or two plugins we need to
> > > > install/tweak etc.
> > > >
> > > > We will be not using 'Views' at the top level, but rather we will
> take
> > > > advantage of 'Folders Plus' - each project will get its own Folder
> and
> > > have
> > > > authorisation access to create/edit jobs etc within that folder. I
> will
> > > > create these folders as you ask for them to start with. This setup
> > allows
> > > > for credentials isolation amongst other benefits, including but not
> > > limited
> > > > to exclusive agents (Controlled Agents) for your own use , should you
> > > have
> > > > any project targeted donations of agents.
> > > >
> > > > As with other aspects of the ASF, projects can choose to just enable
> > all
> > > > committers access to their folder, just ask.
> > > >
> > > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but
> will
> > > not
> > > > be setting up any forwarding rules or anything like that.
> > > >
> > > > So, please, get started *now *on this so you can be sure we are all
> > > > completed before the final cutoff date of 15th August 2020.
> > > >
> > > > Any questions - I expect a few (dozen :) ) - ask away and/or file
> INFRA
> > > > tickets.
> > > >
> > > > Hadoop and related projects have their own migration path to follow,
> > same
> > > > cut off date, Cassandra, Beam, CouchDB have already migrated and are
> > > doing
> > > > very well in their new homes.
> > > >
> > > > Lets get going ...
> > > >
> > > > --
> > > >
> > > > *Gavin McDonald*
> > > > Systems Administrator
> > > > ASF Infrastructure Team
> > >
> > > 2 days until the 15th. Does anyone have access to
> > > https://ci-builds.apache.org/ yet?
> > >
> > > On your mark, get set, go,
> > >
> > > Zach
> > >
> >
>
Re: [EXTERNAL] Re: Migrating to ci-builds.apache.org from builds.apache.org
Posted by Zach Hoffman <za...@zrhoffman.net>.
Yes, GHA lets you persist artifacts, and those artifacts and be whatever
you want. They stick around for 90 days.
-Zach
On Thu, Aug 13, 2020 at 11:28 AM Dave Neuman <ne...@apache.org> wrote:
> Does GHA also host the builds for downloads?
> One nice thing about Jenkins is that it hosts helper builds that we can
> point people in the community to.
On Thu, Aug 13, 2020 at 12:18 PM ocket 8888 <oc...@gmail.com> wrote:
> To be clear, when that PR is merged we won't have "check that each TC
> component builds successfully" or "runs the unit tests agains each
> component". That PR only covers Go unit tests, which means specifically TR
> and TP will be untested still in either build or unit tests.
>
> You actually don't really need CiaB to run the API tests, you just need
> postgres and TO. GHA offers job services, so we can have it load up a
> postgres instance without any weird tricks. Then just run TO in the
> background and run the API tests. I'm working on that as we speak, so far
> the hard part is running the migrations. Because up/down are specified in
> the same file, goose is a real dependency and not just a convenience tool,
> so I gotta get that to run which means building db_admin etc. It's slow,
> but hopefully it'll work fine.
>
> On Thu, Aug 13, 2020 at 11:40 AM Gray, Jonathan <Jonathan_Gray@comcast.com
> >
> wrote:
>
> > I like GHA, but some of our tests (TO API and TP UI tests specifically)
> > are difficult from an OSS CI perspective because of the use of a CIAB
> > sidecar. I realize ocket8888 is making headway on it that's very
> promising
> > in GHA, but at this point we've got an equal to greater belief that ASF
> > infra will be able to handle this situation. We've tinkered with
> replacing
> > our jobs before but due to our usage of the Github PR Builder Jenkins
> > plugin, it's been stalled out. I have a Jenkinsfile made that should be
> > directly portable into the new ASF Jenkins instance done, but our
> cultural
> > usage of that plugin has been a blocker. Now it sounds like our hand is
> > being forced, so I'm +1 on migrating within ASF infrastructure until such
> > time as GHA can cover all our testing use cases.
> >
> > Jonathan G
> >
> >
> > On 8/13/20, 11:16 AM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
> >
> > I think it's time to move away from apache jenkins and embrace github
> > actions (GHA). In addition to providing better visibility on a PR
> (i.e.
> >
> >
> https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4933/checks__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWd6Qdtl2$
> > ), GHAs can be
> > created/modified by non-committers whereas apache jenkins cannot.
> >
> > When this open PR is merged from Brennan:
> >
> >
> https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4948__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWciZPUAx$
> > we will have GHAs that:
> >
> > - run the weasel license checker
> > - check that each TC component builds successfully
> > - runs the unit tests agains each component
> >
> > I "think" this encompasses everything done previously by apache
> > jenkins.
> >
> > +1 for GHA, -1 for migrating to
> >
> https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$
> >
> > Jeremy
> >
> >
> >
> > On Thu, Aug 13, 2020 at 10:36 AM Zach Hoffman <za...@zrhoffman.net>
> > wrote:
> >
> > > Actual link to the builds@apache.org mailing list thread:
> > >
> > >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d*40*3Cbuilds.apache.org*3E__;JSUl!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSD8MssN$
> > >
> > > On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net>
> > wrote:
> > >
> > > > As noted in #dev in the ATC slack yesterday, our Jenkins jobs
> > aren't
> > > > getting run, but the issue isn't a simple "Other projects' jobs
> are
> > > > monopolizing the runners".
> > > >
> > > > If you go to builds.apache.org, there is this banner:
> > > > > IMPORTANT: This Jenkins server will be turned off on the 15th
> > August,
> > > > please migrate your jobs!! See the builds@a.o and/or
> > > > hadoop-migrations@infra.a.o mailing list for more details
> > > >
> > > >
> > > >
> > >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > > > <builds.apache.org>
> > > >
> > > > Initial message from "[IMPORTANT] - Migration to ci-builds.a.o"
> on
> > the
> > > > builds.apache.org mailing list (
> > > >
> > >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > > > <builds.apache.org> ):
> > > >
> > > > > Hi All,
> > > > >
> > > > > This NOTICE is for everyone on builds.apache.org. We are
> > migrating to
> > > a
> > > > new
> > > > > Cloudbees based Client Master called
> >
> https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$
> > . The
> > > > > migrations of all jobs needs to be done before the switch off
> > date of
> > > > 15th
> > > > > August 2020, so you have a maximum of 4 weeks.
> > > > >
> > > > > There is no tool or automated way of migrating your jobs, the
> > > > > differences in the platforms, the plugins and the setup makes
> it
> > > > impossible
> > > > > to do in a safe way. So, you all need to start creating new
> jobs
> > on
> > > > > ci-infra.a.o and then , when you are happy, turn off your old
> > builds on
> > > > > builds.a.o.
> > > > >
> > > > > There are currently 4 agents over there ready to take jobs,
> plus
> > a
> > > > floating
> > > > > agent which is shared amongst many masters (more to come). I
> will
> > > migrate
> > > > > away 2 more agents from builds.a.o to ci-builds.a.o every few
> > days, and
> > > > > will keep an eye of load across both and adjust accordingly.
> > > > >
> > > > > If needed, create a ticket on INFRA jira for any issues that
> > crop up,
> > > or
> > > > > email here on builds@a.o. there may be one or two plugins we
> > need to
> > > > > install/tweak etc.
> > > > >
> > > > > We will be not using 'Views' at the top level, but rather we
> > will take
> > > > > advantage of 'Folders Plus' - each project will get its own
> > Folder and
> > > > have
> > > > > authorisation access to create/edit jobs etc within that
> folder.
> > I will
> > > > > create these folders as you ask for them to start with. This
> > setup
> > > allows
> > > > > for credentials isolation amongst other benefits, including but
> > not
> > > > limited
> > > > > to exclusive agents (Controlled Agents) for your own use ,
> > should you
> > > > have
> > > > > any project targeted donations of agents.
> > > > >
> > > > > As with other aspects of the ASF, projects can choose to just
> > enable
> > > all
> > > > > committers access to their folder, just ask.
> > > > >
> > > > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o
> > but will
> > > > not
> > > > > be setting up any forwarding rules or anything like that.
> > > > >
> > > > > So, please, get started *now *on this so you can be sure we are
> > all
> > > > > completed before the final cutoff date of 15th August 2020.
> > > > >
> > > > > Any questions - I expect a few (dozen :) ) - ask away and/or
> > file INFRA
> > > > > tickets.
> > > > >
> > > > > Hadoop and related projects have their own migration path to
> > follow,
> > > same
> > > > > cut off date, Cassandra, Beam, CouchDB have already migrated
> and
> > are
> > > > doing
> > > > > very well in their new homes.
> > > > >
> > > > > Lets get going ...
> > > > >
> > > > > --
> > > > >
> > > > > *Gavin McDonald*
> > > > > Systems Administrator
> > > > > ASF Infrastructure Team
> > > >
> > > > 2 days until the 15th. Does anyone have access to
> > > >
> >
> https://urldefense.com/v3/__https://ci-builds.apache.org/__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSx3XDtN$
> > yet?
> > > >
> > > > On your mark, get set, go,
> > > >
> > > > Zach
> > > >
> > >
> >
> >
>
Re: [EXTERNAL] Re: Migrating to ci-builds.apache.org from builds.apache.org
Posted by ocket 8888 <oc...@gmail.com>.
To be clear, when that PR is merged we won't have "check that each TC
component builds successfully" or "runs the unit tests agains each
component". That PR only covers Go unit tests, which means specifically TR
and TP will be untested still in either build or unit tests.
You actually don't really need CiaB to run the API tests, you just need
postgres and TO. GHA offers job services, so we can have it load up a
postgres instance without any weird tricks. Then just run TO in the
background and run the API tests. I'm working on that as we speak, so far
the hard part is running the migrations. Because up/down are specified in
the same file, goose is a real dependency and not just a convenience tool,
so I gotta get that to run which means building db_admin etc. It's slow,
but hopefully it'll work fine.
On Thu, Aug 13, 2020 at 11:40 AM Gray, Jonathan <Jo...@comcast.com>
wrote:
> I like GHA, but some of our tests (TO API and TP UI tests specifically)
> are difficult from an OSS CI perspective because of the use of a CIAB
> sidecar. I realize ocket8888 is making headway on it that's very promising
> in GHA, but at this point we've got an equal to greater belief that ASF
> infra will be able to handle this situation. We've tinkered with replacing
> our jobs before but due to our usage of the Github PR Builder Jenkins
> plugin, it's been stalled out. I have a Jenkinsfile made that should be
> directly portable into the new ASF Jenkins instance done, but our cultural
> usage of that plugin has been a blocker. Now it sounds like our hand is
> being forced, so I'm +1 on migrating within ASF infrastructure until such
> time as GHA can cover all our testing use cases.
>
> Jonathan G
>
>
> On 8/13/20, 11:16 AM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
>
> I think it's time to move away from apache jenkins and embrace github
> actions (GHA). In addition to providing better visibility on a PR (i.e.
>
> https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4933/checks__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWd6Qdtl2$
> ), GHAs can be
> created/modified by non-committers whereas apache jenkins cannot.
>
> When this open PR is merged from Brennan:
>
> https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4948__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWciZPUAx$
> we will have GHAs that:
>
> - run the weasel license checker
> - check that each TC component builds successfully
> - runs the unit tests agains each component
>
> I "think" this encompasses everything done previously by apache
> jenkins.
>
> +1 for GHA, -1 for migrating to
> https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$
>
> Jeremy
>
>
>
> On Thu, Aug 13, 2020 at 10:36 AM Zach Hoffman <za...@zrhoffman.net>
> wrote:
>
> > Actual link to the builds@apache.org mailing list thread:
> >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d*40*3Cbuilds.apache.org*3E__;JSUl!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSD8MssN$
> >
> > On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net>
> wrote:
> >
> > > As noted in #dev in the ATC slack yesterday, our Jenkins jobs
> aren't
> > > getting run, but the issue isn't a simple "Other projects' jobs are
> > > monopolizing the runners".
> > >
> > > If you go to builds.apache.org, there is this banner:
> > > > IMPORTANT: This Jenkins server will be turned off on the 15th
> August,
> > > please migrate your jobs!! See the builds@a.o and/or
> > > hadoop-migrations@infra.a.o mailing list for more details
> > >
> > >
> > >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > > <builds.apache.org>
> > >
> > > Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on
> the
> > > builds.apache.org mailing list (
> > >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > > <builds.apache.org> ):
> > >
> > > > Hi All,
> > > >
> > > > This NOTICE is for everyone on builds.apache.org. We are
> migrating to
> > a
> > > new
> > > > Cloudbees based Client Master called
> https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$
> . The
> > > > migrations of all jobs needs to be done before the switch off
> date of
> > > 15th
> > > > August 2020, so you have a maximum of 4 weeks.
> > > >
> > > > There is no tool or automated way of migrating your jobs, the
> > > > differences in the platforms, the plugins and the setup makes it
> > > impossible
> > > > to do in a safe way. So, you all need to start creating new jobs
> on
> > > > ci-infra.a.o and then , when you are happy, turn off your old
> builds on
> > > > builds.a.o.
> > > >
> > > > There are currently 4 agents over there ready to take jobs, plus
> a
> > > floating
> > > > agent which is shared amongst many masters (more to come). I will
> > migrate
> > > > away 2 more agents from builds.a.o to ci-builds.a.o every few
> days, and
> > > > will keep an eye of load across both and adjust accordingly.
> > > >
> > > > If needed, create a ticket on INFRA jira for any issues that
> crop up,
> > or
> > > > email here on builds@a.o. there may be one or two plugins we
> need to
> > > > install/tweak etc.
> > > >
> > > > We will be not using 'Views' at the top level, but rather we
> will take
> > > > advantage of 'Folders Plus' - each project will get its own
> Folder and
> > > have
> > > > authorisation access to create/edit jobs etc within that folder.
> I will
> > > > create these folders as you ask for them to start with. This
> setup
> > allows
> > > > for credentials isolation amongst other benefits, including but
> not
> > > limited
> > > > to exclusive agents (Controlled Agents) for your own use ,
> should you
> > > have
> > > > any project targeted donations of agents.
> > > >
> > > > As with other aspects of the ASF, projects can choose to just
> enable
> > all
> > > > committers access to their folder, just ask.
> > > >
> > > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o
> but will
> > > not
> > > > be setting up any forwarding rules or anything like that.
> > > >
> > > > So, please, get started *now *on this so you can be sure we are
> all
> > > > completed before the final cutoff date of 15th August 2020.
> > > >
> > > > Any questions - I expect a few (dozen :) ) - ask away and/or
> file INFRA
> > > > tickets.
> > > >
> > > > Hadoop and related projects have their own migration path to
> follow,
> > same
> > > > cut off date, Cassandra, Beam, CouchDB have already migrated and
> are
> > > doing
> > > > very well in their new homes.
> > > >
> > > > Lets get going ...
> > > >
> > > > --
> > > >
> > > > *Gavin McDonald*
> > > > Systems Administrator
> > > > ASF Infrastructure Team
> > >
> > > 2 days until the 15th. Does anyone have access to
> > >
> https://urldefense.com/v3/__https://ci-builds.apache.org/__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSx3XDtN$
> yet?
> > >
> > > On your mark, get set, go,
> > >
> > > Zach
> > >
> >
>
>
Re: [EXTERNAL] Re: Migrating to ci-builds.apache.org from
builds.apache.org
Posted by "Gray, Jonathan" <Jo...@comcast.com>.
I like GHA, but some of our tests (TO API and TP UI tests specifically) are difficult from an OSS CI perspective because of the use of a CIAB sidecar. I realize ocket8888 is making headway on it that's very promising in GHA, but at this point we've got an equal to greater belief that ASF infra will be able to handle this situation. We've tinkered with replacing our jobs before but due to our usage of the Github PR Builder Jenkins plugin, it's been stalled out. I have a Jenkinsfile made that should be directly portable into the new ASF Jenkins instance done, but our cultural usage of that plugin has been a blocker. Now it sounds like our hand is being forced, so I'm +1 on migrating within ASF infrastructure until such time as GHA can cover all our testing use cases.
Jonathan G
On 8/13/20, 11:16 AM, "Jeremy Mitchell" <mi...@gmail.com> wrote:
I think it's time to move away from apache jenkins and embrace github
actions (GHA). In addition to providing better visibility on a PR (i.e.
https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4933/checks__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWd6Qdtl2$ ), GHAs can be
created/modified by non-committers whereas apache jenkins cannot.
When this open PR is merged from Brennan:
https://urldefense.com/v3/__https://github.com/apache/trafficcontrol/pull/4948__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWciZPUAx$ we will have GHAs that:
- run the weasel license checker
- check that each TC component builds successfully
- runs the unit tests agains each component
I "think" this encompasses everything done previously by apache jenkins.
+1 for GHA, -1 for migrating to https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$
Jeremy
On Thu, Aug 13, 2020 at 10:36 AM Zach Hoffman <za...@zrhoffman.net> wrote:
> Actual link to the builds@apache.org mailing list thread:
>
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d*40*3Cbuilds.apache.org*3E__;JSUl!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSD8MssN$
>
> On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net> wrote:
>
> > As noted in #dev in the ATC slack yesterday, our Jenkins jobs aren't
> > getting run, but the issue isn't a simple "Other projects' jobs are
> > monopolizing the runners".
> >
> > If you go to builds.apache.org, there is this banner:
> > > IMPORTANT: This Jenkins server will be turned off on the 15th August,
> > please migrate your jobs!! See the builds@a.o and/or
> > hadoop-migrations@infra.a.o mailing list for more details
> >
> >
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > <builds.apache.org>
> >
> > Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on the
> > builds.apache.org mailing list (
> >
> https://urldefense.com/v3/__https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWTrMjSRj$
> > <builds.apache.org> ):
> >
> > > Hi All,
> > >
> > > This NOTICE is for everyone on builds.apache.org. We are migrating to
> a
> > new
> > > Cloudbees based Client Master called https://urldefense.com/v3/__https://ci-builds.apache.org__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWYyfZLAN$ . The
> > > migrations of all jobs needs to be done before the switch off date of
> > 15th
> > > August 2020, so you have a maximum of 4 weeks.
> > >
> > > There is no tool or automated way of migrating your jobs, the
> > > differences in the platforms, the plugins and the setup makes it
> > impossible
> > > to do in a safe way. So, you all need to start creating new jobs on
> > > ci-infra.a.o and then , when you are happy, turn off your old builds on
> > > builds.a.o.
> > >
> > > There are currently 4 agents over there ready to take jobs, plus a
> > floating
> > > agent which is shared amongst many masters (more to come). I will
> migrate
> > > away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> > > will keep an eye of load across both and adjust accordingly.
> > >
> > > If needed, create a ticket on INFRA jira for any issues that crop up,
> or
> > > email here on builds@a.o. there may be one or two plugins we need to
> > > install/tweak etc.
> > >
> > > We will be not using 'Views' at the top level, but rather we will take
> > > advantage of 'Folders Plus' - each project will get its own Folder and
> > have
> > > authorisation access to create/edit jobs etc within that folder. I will
> > > create these folders as you ask for them to start with. This setup
> allows
> > > for credentials isolation amongst other benefits, including but not
> > limited
> > > to exclusive agents (Controlled Agents) for your own use , should you
> > have
> > > any project targeted donations of agents.
> > >
> > > As with other aspects of the ASF, projects can choose to just enable
> all
> > > committers access to their folder, just ask.
> > >
> > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> > not
> > > be setting up any forwarding rules or anything like that.
> > >
> > > So, please, get started *now *on this so you can be sure we are all
> > > completed before the final cutoff date of 15th August 2020.
> > >
> > > Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> > > tickets.
> > >
> > > Hadoop and related projects have their own migration path to follow,
> same
> > > cut off date, Cassandra, Beam, CouchDB have already migrated and are
> > doing
> > > very well in their new homes.
> > >
> > > Lets get going ...
> > >
> > > --
> > >
> > > *Gavin McDonald*
> > > Systems Administrator
> > > ASF Infrastructure Team
> >
> > 2 days until the 15th. Does anyone have access to
> > https://urldefense.com/v3/__https://ci-builds.apache.org/__;!!CQl3mcHX2A!UsjHewB4L2xwQJfJ9O7F9acjWSZ6uXV03HbFEz6JvvgUMgZ_ywazU6i1zJLYWSx3XDtN$ yet?
> >
> > On your mark, get set, go,
> >
> > Zach
> >
>
Re: Migrating to ci-builds.apache.org from builds.apache.org
Posted by Jeremy Mitchell <mi...@gmail.com>.
I think it's time to move away from apache jenkins and embrace github
actions (GHA). In addition to providing better visibility on a PR (i.e.
https://github.com/apache/trafficcontrol/pull/4933/checks), GHAs can be
created/modified by non-committers whereas apache jenkins cannot.
When this open PR is merged from Brennan:
https://github.com/apache/trafficcontrol/pull/4948 we will have GHAs that:
- run the weasel license checker
- check that each TC component builds successfully
- runs the unit tests agains each component
I "think" this encompasses everything done previously by apache jenkins.
+1 for GHA, -1 for migrating to https://ci-builds.apache.org
Jeremy
On Thu, Aug 13, 2020 at 10:36 AM Zach Hoffman <za...@zrhoffman.net> wrote:
> Actual link to the builds@apache.org mailing list thread:
>
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d%40%3Cbuilds.apache.org%3E
>
> On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net> wrote:
>
> > As noted in #dev in the ATC slack yesterday, our Jenkins jobs aren't
> > getting run, but the issue isn't a simple "Other projects' jobs are
> > monopolizing the runners".
> >
> > If you go to builds.apache.org, there is this banner:
> > > IMPORTANT: This Jenkins server will be turned off on the 15th August,
> > please migrate your jobs!! See the builds@a.o and/or
> > hadoop-migrations@infra.a.o mailing list for more details
> >
> >
> >
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> > <builds.apache.org>
> >
> > Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on the
> > builds.apache.org mailing list (
> >
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> > <builds.apache.org> ):
> >
> > > Hi All,
> > >
> > > This NOTICE is for everyone on builds.apache.org. We are migrating to
> a
> > new
> > > Cloudbees based Client Master called https://ci-builds.apache.org. The
> > > migrations of all jobs needs to be done before the switch off date of
> > 15th
> > > August 2020, so you have a maximum of 4 weeks.
> > >
> > > There is no tool or automated way of migrating your jobs, the
> > > differences in the platforms, the plugins and the setup makes it
> > impossible
> > > to do in a safe way. So, you all need to start creating new jobs on
> > > ci-infra.a.o and then , when you are happy, turn off your old builds on
> > > builds.a.o.
> > >
> > > There are currently 4 agents over there ready to take jobs, plus a
> > floating
> > > agent which is shared amongst many masters (more to come). I will
> migrate
> > > away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> > > will keep an eye of load across both and adjust accordingly.
> > >
> > > If needed, create a ticket on INFRA jira for any issues that crop up,
> or
> > > email here on builds@a.o. there may be one or two plugins we need to
> > > install/tweak etc.
> > >
> > > We will be not using 'Views' at the top level, but rather we will take
> > > advantage of 'Folders Plus' - each project will get its own Folder and
> > have
> > > authorisation access to create/edit jobs etc within that folder. I will
> > > create these folders as you ask for them to start with. This setup
> allows
> > > for credentials isolation amongst other benefits, including but not
> > limited
> > > to exclusive agents (Controlled Agents) for your own use , should you
> > have
> > > any project targeted donations of agents.
> > >
> > > As with other aspects of the ASF, projects can choose to just enable
> all
> > > committers access to their folder, just ask.
> > >
> > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> > not
> > > be setting up any forwarding rules or anything like that.
> > >
> > > So, please, get started *now *on this so you can be sure we are all
> > > completed before the final cutoff date of 15th August 2020.
> > >
> > > Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> > > tickets.
> > >
> > > Hadoop and related projects have their own migration path to follow,
> same
> > > cut off date, Cassandra, Beam, CouchDB have already migrated and are
> > doing
> > > very well in their new homes.
> > >
> > > Lets get going ...
> > >
> > > --
> > >
> > > *Gavin McDonald*
> > > Systems Administrator
> > > ASF Infrastructure Team
> >
> > 2 days until the 15th. Does anyone have access to
> > https://ci-builds.apache.org/ yet?
> >
> > On your mark, get set, go,
> >
> > Zach
> >
>
Re: Migrating to ci-builds.apache.org from builds.apache.org
Posted by Zach Hoffman <za...@zrhoffman.net>.
Actual link to the builds@apache.org mailing list thread:
https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d%40%3Cbuilds.apache.org%3E
On Thu, Aug 13, 2020 at 10:34 AM Zach Hoffman <za...@zrhoffman.net> wrote:
> As noted in #dev in the ATC slack yesterday, our Jenkins jobs aren't
> getting run, but the issue isn't a simple "Other projects' jobs are
> monopolizing the runners".
>
> If you go to builds.apache.org, there is this banner:
> > IMPORTANT: This Jenkins server will be turned off on the 15th August,
> please migrate your jobs!! See the builds@a.o and/or
> hadoop-migrations@infra.a.o mailing list for more details
>
>
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> <builds.apache.org>
>
> Initial message from "[IMPORTANT] - Migration to ci-builds.a.o" on the
> builds.apache.org mailing list (
> https://lists.apache.org/thread.html/re974eed417a1bc294694701d5c91b4bf92689fcf32a4c91f169be87d@
> <builds.apache.org> ):
>
> > Hi All,
> >
> > This NOTICE is for everyone on builds.apache.org. We are migrating to a
> new
> > Cloudbees based Client Master called https://ci-builds.apache.org. The
> > migrations of all jobs needs to be done before the switch off date of
> 15th
> > August 2020, so you have a maximum of 4 weeks.
> >
> > There is no tool or automated way of migrating your jobs, the
> > differences in the platforms, the plugins and the setup makes it
> impossible
> > to do in a safe way. So, you all need to start creating new jobs on
> > ci-infra.a.o and then , when you are happy, turn off your old builds on
> > builds.a.o.
> >
> > There are currently 4 agents over there ready to take jobs, plus a
> floating
> > agent which is shared amongst many masters (more to come). I will migrate
> > away 2 more agents from builds.a.o to ci-builds.a.o every few days, and
> > will keep an eye of load across both and adjust accordingly.
> >
> > If needed, create a ticket on INFRA jira for any issues that crop up, or
> > email here on builds@a.o. there may be one or two plugins we need to
> > install/tweak etc.
> >
> > We will be not using 'Views' at the top level, but rather we will take
> > advantage of 'Folders Plus' - each project will get its own Folder and
> have
> > authorisation access to create/edit jobs etc within that folder. I will
> > create these folders as you ask for them to start with. This setup allows
> > for credentials isolation amongst other benefits, including but not
> limited
> > to exclusive agents (Controlled Agents) for your own use , should you
> have
> > any project targeted donations of agents.
> >
> > As with other aspects of the ASF, projects can choose to just enable all
> > committers access to their folder, just ask.
> >
> > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will
> not
> > be setting up any forwarding rules or anything like that.
> >
> > So, please, get started *now *on this so you can be sure we are all
> > completed before the final cutoff date of 15th August 2020.
> >
> > Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA
> > tickets.
> >
> > Hadoop and related projects have their own migration path to follow, same
> > cut off date, Cassandra, Beam, CouchDB have already migrated and are
> doing
> > very well in their new homes.
> >
> > Lets get going ...
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
>
> 2 days until the 15th. Does anyone have access to
> https://ci-builds.apache.org/ yet?
>
> On your mark, get set, go,
>
> Zach
>