Re: [DISCUSS] Jenkins -> GitHub Actions ?

[Austin Bennett <au...@apache.org>]

+1

Also would help address a good amount of what concerns me that was [sorta]
raised by https://lists.apache.org/thread/7jr99nc5xsb3ft1d75kb0ml32bzw89rv


Once we think this is something we want to do, but might be
blocked/concerned because of lack of definitively comparable features, I'd
be happy to take a look at what exists in the wider ecosystem or could be
built.

Cheers -



On Fri, Oct 21, 2022 at 11:10 AM Ismaël Mejía <ie...@gmail.com> wrote:

> +1 Github Actions are more intuitive and easy to modify and test for
> everyone.
> Also Beam wins because that makes one less system to maintain.
>
> Regards,
> Ismaël
>
> On Wed, Oct 19, 2022 at 5:50 PM Danny McCormick via dev
> <de...@beam.apache.org> wrote:
> >
> > Thanks for kicking this conversation off. I'm +1 on migrating, but only
> once we've found a specific replacement for easy observability (which
> workflows have been failing lately, and how often) and trigger phrases (for
> retries and workflows that aren't automatically kicked off but should be
> run for extra validation, e.g. postcommits). Until we have viable
> replacements, I don't think we should make the move. Publishing nightly
> snapshots is eventually also a must to fully migrate, but probably doesn't
> need to block us from making progress here.
> >
> > With those caveats, the reason that I'm +1 on moving is that our Jenkins
> reliability has been rough. Since I joined the project in January, I can
> think of 3 different incidents that significantly harmed our ability to do
> work.
> >
> > 1. Jenkins triggers cause multi-day outage - this led to a multi-day
> code freeze, and we lost our trigger functionality for days afterwards.
> Investigating/restoring our state ate up a pretty full week for me.
> > 2. Jenkins plugin cause multi-day outage - this led to multiple days of
> Jenkins downtime before eventually being resolved by Infra.
> > 3. Cert issues cause many workers to go down - I don't have a thread for
> this because I handled most of the investigation the day of, but many of
> our workers went down for around a day and nobody noticed until queue time
> reached 6+ hours for each workflow.
> >
> > There may be others that I'm overlooking.
> >
> > GitHub Actions isn't a magic bullet to fix these problems, but it
> minimizes the amount of infra that we're maintaining ourselves, increases
> the isolation between workflows (catastrophic failure is less likely), has
> uptime guarantees, and is more likely to receive investment going forward
> (we're likely to get increasing benefits over time for free). We've also
> done a lot of exploration in this area already, so we're not starting from
> scratch.
> >
> > Thanks,
> > Danny
> >
> > On Wed, Oct 19, 2022 at 11:32 AM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>
> >> Hi all,
> >>
> >> As you probably noticed, there's a lot of work going on around adding
> more GitHub Actions workflows.
> >>
> >> Can we fully migrate to GitHub Actions? Similar to our GitHub Issues
> migration (but less user-facing) it would bring us on to "default"
> infrastructure that more people understand and is maintained by GitHub.
> >>
> >> So far we have hit some serious roadblocks. It isn't just a simple
> migration. We have to weigh doing the work to get there.
> >>
> >> I started a document with a table of the things we get from Jenkins
> that we need to be sure to have for GitHub Actions before we could think
> about migrating:
> >>
> >> https://s.apache.org/beam-jenkins-to-gha
> >>
> >> Can you please help me by adding things that we get from Jenkins, and
> if you know how to get them from GitHub Actions add that too.
> >>
> >> Thanks!
> >>
> >> Kenn
>

Re: [DISCUSS] Jenkins -> GitHub Actions ?

[Tomo Suzuki via dev <de...@beam.apache.org>]

Kenn, thank you for the summary. SGTM. Looking forward to GitHub Actions.

On Mon, Nov 7, 2022 at 12:58 PM Kenneth Knowles <ke...@apache.org> wrote:

> OK, it seems like there is general consensus. Not too much action on the
> document. I will summarize the gaps that don't have an answer in the doc,
> and my new opinion of how important they are:
>
>  - [required] Run specific non-default workflow on PR
>  - [required] View history of a workflow
>  - [required] Publish nightly snapshots
>  - [required] Run workflow on dedicated worker pool for performance testing
>  - [important but not required] Summarize flakiness statistics of one or
> all workflows
>  - [important but not required] History of all/many workflows in a single
> view
>  - [nice to have] History of specific test case (not just the workflow
> level)
>
> Do any of these seem like I got the importance wrong?
>
> Kenn
>
> On Mon, Nov 7, 2022 at 9:09 AM Austin Bennett <au...@apache.org> wrote:
>
>> +1
>>
>> Also would help address a good amount of what concerns me that was
>> [sorta] raised by
>> https://lists.apache.org/thread/7jr99nc5xsb3ft1d75kb0ml32bzw89rv
>>
>>
>> Once we think this is something we want to do, but might be
>> blocked/concerned because of lack of definitively comparable features, I'd
>> be happy to take a look at what exists in the wider ecosystem or could be
>> built.
>>
>> Cheers -
>>
>>
>>
>> On Fri, Oct 21, 2022 at 11:10 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> +1 Github Actions are more intuitive and easy to modify and test for
>>> everyone.
>>> Also Beam wins because that makes one less system to maintain.
>>>
>>> Regards,
>>> Ismaël
>>>
>>> On Wed, Oct 19, 2022 at 5:50 PM Danny McCormick via dev
>>> <de...@beam.apache.org> wrote:
>>> >
>>> > Thanks for kicking this conversation off. I'm +1 on migrating, but
>>> only once we've found a specific replacement for easy observability (which
>>> workflows have been failing lately, and how often) and trigger phrases (for
>>> retries and workflows that aren't automatically kicked off but should be
>>> run for extra validation, e.g. postcommits). Until we have viable
>>> replacements, I don't think we should make the move. Publishing nightly
>>> snapshots is eventually also a must to fully migrate, but probably doesn't
>>> need to block us from making progress here.
>>> >
>>> > With those caveats, the reason that I'm +1 on moving is that our
>>> Jenkins reliability has been rough. Since I joined the project in January,
>>> I can think of 3 different incidents that significantly harmed our ability
>>> to do work.
>>> >
>>> > 1. Jenkins triggers cause multi-day outage - this led to a multi-day
>>> code freeze, and we lost our trigger functionality for days afterwards.
>>> Investigating/restoring our state ate up a pretty full week for me.
>>> > 2. Jenkins plugin cause multi-day outage - this led to multiple days
>>> of Jenkins downtime before eventually being resolved by Infra.
>>> > 3. Cert issues cause many workers to go down - I don't have a thread
>>> for this because I handled most of the investigation the day of, but many
>>> of our workers went down for around a day and nobody noticed until queue
>>> time reached 6+ hours for each workflow.
>>> >
>>> > There may be others that I'm overlooking.
>>> >
>>> > GitHub Actions isn't a magic bullet to fix these problems, but it
>>> minimizes the amount of infra that we're maintaining ourselves, increases
>>> the isolation between workflows (catastrophic failure is less likely), has
>>> uptime guarantees, and is more likely to receive investment going forward
>>> (we're likely to get increasing benefits over time for free). We've also
>>> done a lot of exploration in this area already, so we're not starting from
>>> scratch.
>>> >
>>> > Thanks,
>>> > Danny
>>> >
>>> > On Wed, Oct 19, 2022 at 11:32 AM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> As you probably noticed, there's a lot of work going on around adding
>>> more GitHub Actions workflows.
>>> >>
>>> >> Can we fully migrate to GitHub Actions? Similar to our GitHub Issues
>>> migration (but less user-facing) it would bring us on to "default"
>>> infrastructure that more people understand and is maintained by GitHub.
>>> >>
>>> >> So far we have hit some serious roadblocks. It isn't just a simple
>>> migration. We have to weigh doing the work to get there.
>>> >>
>>> >> I started a document with a table of the things we get from Jenkins
>>> that we need to be sure to have for GitHub Actions before we could think
>>> about migrating:
>>> >>
>>> >> https://s.apache.org/beam-jenkins-to-gha
>>> >>
>>> >> Can you please help me by adding things that we get from Jenkins, and
>>> if you know how to get them from GitHub Actions add that too.
>>> >>
>>> >> Thanks!
>>> >>
>>> >> Kenn
>>>
>>

-- 
Regards,
Tomo

Re: [DISCUSS] Jenkins -> GitHub Actions ?

[Kenneth Knowles <ke...@apache.org>]

OK, it seems like there is general consensus. Not too much action on the
document. I will summarize the gaps that don't have an answer in the doc,
and my new opinion of how important they are:

 - [required] Run specific non-default workflow on PR
 - [required] View history of a workflow
 - [required] Publish nightly snapshots
 - [required] Run workflow on dedicated worker pool for performance testing
 - [important but not required] Summarize flakiness statistics of one or
all workflows
 - [important but not required] History of all/many workflows in a single
view
 - [nice to have] History of specific test case (not just the workflow
level)

Do any of these seem like I got the importance wrong?

Kenn

On Mon, Nov 7, 2022 at 9:09 AM Austin Bennett <au...@apache.org> wrote:

> +1
>
> Also would help address a good amount of what concerns me that was [sorta]
> raised by https://lists.apache.org/thread/7jr99nc5xsb3ft1d75kb0ml32bzw89rv
>
>
>
> Once we think this is something we want to do, but might be
> blocked/concerned because of lack of definitively comparable features, I'd
> be happy to take a look at what exists in the wider ecosystem or could be
> built.
>
> Cheers -
>
>
>
> On Fri, Oct 21, 2022 at 11:10 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> +1 Github Actions are more intuitive and easy to modify and test for
>> everyone.
>> Also Beam wins because that makes one less system to maintain.
>>
>> Regards,
>> Ismaël
>>
>> On Wed, Oct 19, 2022 at 5:50 PM Danny McCormick via dev
>> <de...@beam.apache.org> wrote:
>> >
>> > Thanks for kicking this conversation off. I'm +1 on migrating, but only
>> once we've found a specific replacement for easy observability (which
>> workflows have been failing lately, and how often) and trigger phrases (for
>> retries and workflows that aren't automatically kicked off but should be
>> run for extra validation, e.g. postcommits). Until we have viable
>> replacements, I don't think we should make the move. Publishing nightly
>> snapshots is eventually also a must to fully migrate, but probably doesn't
>> need to block us from making progress here.
>> >
>> > With those caveats, the reason that I'm +1 on moving is that our
>> Jenkins reliability has been rough. Since I joined the project in January,
>> I can think of 3 different incidents that significantly harmed our ability
>> to do work.
>> >
>> > 1. Jenkins triggers cause multi-day outage - this led to a multi-day
>> code freeze, and we lost our trigger functionality for days afterwards.
>> Investigating/restoring our state ate up a pretty full week for me.
>> > 2. Jenkins plugin cause multi-day outage - this led to multiple days of
>> Jenkins downtime before eventually being resolved by Infra.
>> > 3. Cert issues cause many workers to go down - I don't have a thread
>> for this because I handled most of the investigation the day of, but many
>> of our workers went down for around a day and nobody noticed until queue
>> time reached 6+ hours for each workflow.
>> >
>> > There may be others that I'm overlooking.
>> >
>> > GitHub Actions isn't a magic bullet to fix these problems, but it
>> minimizes the amount of infra that we're maintaining ourselves, increases
>> the isolation between workflows (catastrophic failure is less likely), has
>> uptime guarantees, and is more likely to receive investment going forward
>> (we're likely to get increasing benefits over time for free). We've also
>> done a lot of exploration in this area already, so we're not starting from
>> scratch.
>> >
>> > Thanks,
>> > Danny
>> >
>> > On Wed, Oct 19, 2022 at 11:32 AM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> As you probably noticed, there's a lot of work going on around adding
>> more GitHub Actions workflows.
>> >>
>> >> Can we fully migrate to GitHub Actions? Similar to our GitHub Issues
>> migration (but less user-facing) it would bring us on to "default"
>> infrastructure that more people understand and is maintained by GitHub.
>> >>
>> >> So far we have hit some serious roadblocks. It isn't just a simple
>> migration. We have to weigh doing the work to get there.
>> >>
>> >> I started a document with a table of the things we get from Jenkins
>> that we need to be sure to have for GitHub Actions before we could think
>> about migrating:
>> >>
>> >> https://s.apache.org/beam-jenkins-to-gha
>> >>
>> >> Can you please help me by adding things that we get from Jenkins, and
>> if you know how to get them from GitHub Actions add that too.
>> >>
>> >> Thanks!
>> >>
>> >> Kenn
>>
>