You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Michael Bien <mb...@gmail.com> on 2022/08/10 17:22:08 UTC
running CI jobs based on PR labels
Hello devs,
while migrating some jobs from travis to github CI, I started making
some (time consuming) jobs and/or build steps conditional based on PR
labels.
I keep it here short since most of the info is in the PR text, but as
example:
The new test job called "Build Tools" would only run if the PR is
labeled with Ant, Gradle, Maven or MX.
There are also additional command labels available, ci:all-tests would
enable everything, while ci:no-build would as you would guess disable
everything (e.g if all what the PR does is to edit a readme or sync-PRs
which syncs two branches could use this too, since they build everything
twice right now).
This is just about PRs, the merge will always test everything on master.
Why labels and not path based dependency checks? Because they never work
outside of fairly trivial cases in my experience. They either test too
much or not enough and whats worse: the dev has to second guess if the
right things are actually being tested - if they aren't there must be
also a second mechanism to override the automation.
With labels, reviewers can decide what needs to be tested based on
experience. A correctly labeled PR should "just work" most of the time
even if you wouldn't know about the details. However, everyone still has
to know about it: If you want the right things to be tested, you have to
label the PR _before_ pressing the create button, otherwise the first
run will have the wrong settings. (syncs will pick up new labels)
I was considering to fail the build for PRs without labels (not
implemented).
https://github.com/apache/netbeans/pull/4431
feedback appreciated,
michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 17.08.22 08:53, Neil C Smith wrote:
> On Wed, 17 Aug 2022 at 04:10, Michael Bien <mb...@gmail.com> wrote:
>>>> OK, this is probably due to
>>>> https://github.com/apache/netbeans/pull/3932 I guess we need to group
>>>> by head and base ref?
>>> yeahhh... maybe. This is making my head smoke thinking about all
>>> scenarios this workflow has to deal with.
>> Played with it a bit in a private repo and simply concatenating the
>> base_ref might indeed work (even without having a ternary operator):
>>
>> group: ${{ github.head_ref || github.run_id }}-${{ github.base_ref }}
>>
>> non PR:
>>
>> "1234-"
>>
>> PRs:
>>
>> "head_ref-base_ref"
> Yes, that's what I had in mind. Possibly simpler - wasn't 100% sure
> what the undefined output was - empty string is good enough.
https://github.com/apache/netbeans/pull/4514
there doesn't seem to be a field for PR numbers, otherwise this would be
a simple:
pr_num || run_id
since thats essentially what we want I guess.
-mbien
>
> Thanks,
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Neil C Smith <ne...@apache.org>.
On Wed, 17 Aug 2022 at 04:10, Michael Bien <mb...@gmail.com> wrote:
> >> OK, this is probably due to
> >> https://github.com/apache/netbeans/pull/3932 I guess we need to group
> >> by head and base ref?
> >
> > yeahhh... maybe. This is making my head smoke thinking about all
> > scenarios this workflow has to deal with.
>
> Played with it a bit in a private repo and simply concatenating the
> base_ref might indeed work (even without having a ternary operator):
>
> group: ${{ github.head_ref || github.run_id }}-${{ github.base_ref }}
>
> non PR:
>
> "1234-"
>
> PRs:
>
> "head_ref-base_ref"
Yes, that's what I had in mind. Possibly simpler - wasn't 100% sure
what the undefined output was - empty string is good enough.
Thanks,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 16.08.22 17:23, Michael Bien wrote:
> On 16.08.22 12:20, Neil C Smith wrote:
>> On Tue, 16 Aug 2022 at 09:25, Neil C Smith <ne...@apache.org>
>> wrote:
>>> On Thu, 11 Aug 2022 at 14:20, Michael Bien <mb...@gmail.com> wrote:
>>>>> The branch tests run on the head of the delivery branch
>>>>> (refs/head/delivery). The sync PR tests run on the pending result of
>>>>> the merge to the base branch (refs/pull/<ID>/merge), eg. master.
>>> The above assumption might need checking. Just restarting a
>>> pull-request test on the sync PRs and noticed that it restarted in
>>> both - ie. two sync PRs with different base branches share a single
>>> set of tests on both GitHub and Travis. At the moment, it's the
>>> result of the merge to master, so the important one. Not sure how
>>> consistent the behaviour is - workflow history shows the other one
>>> cancelled.
>> OK, this is probably due to
>> https://github.com/apache/netbeans/pull/3932 I guess we need to group
>> by head and base ref?
>
> yeahhh... maybe. This is making my head smoke thinking about all
> scenarios this workflow has to deal with.
Played with it a bit in a private repo and simply concatenating the
base_ref might indeed work (even without having a ternary operator):
group: ${{ github.head_ref || github.run_id }}-${{ github.base_ref }}
non PR:
"1234-"
PRs:
"head_ref-base_ref"
-> two sync PRs syncing two distinct branches won't cancel each other
anymore.
-mbien
>
>> Neil
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>
>> For further information about the NetBeans mailing lists, visit:
>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 16.08.22 12:20, Neil C Smith wrote:
> On Tue, 16 Aug 2022 at 09:25, Neil C Smith <ne...@apache.org> wrote:
>> On Thu, 11 Aug 2022 at 14:20, Michael Bien <mb...@gmail.com> wrote:
>>>> The branch tests run on the head of the delivery branch
>>>> (refs/head/delivery). The sync PR tests run on the pending result of
>>>> the merge to the base branch (refs/pull/<ID>/merge), eg. master.
>> The above assumption might need checking. Just restarting a
>> pull-request test on the sync PRs and noticed that it restarted in
>> both - ie. two sync PRs with different base branches share a single
>> set of tests on both GitHub and Travis. At the moment, it's the
>> result of the merge to master, so the important one. Not sure how
>> consistent the behaviour is - workflow history shows the other one
>> cancelled.
> OK, this is probably due to
> https://github.com/apache/netbeans/pull/3932 I guess we need to group
> by head and base ref?
yeahhh... maybe. This is making my head smoke thinking about all
scenarios this workflow has to deal with.
current logic is really simple:
if there is a github.head_ref
cancel other run with the same head_ref if there is one
start this run and group it as head_ref
else
group by github.run_id which is an UID which means "always run"
you can probably just restart one workflow/job if the other finishes as
workaround, right? :)
-mbien
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Neil C Smith <ne...@apache.org>.
On Tue, 16 Aug 2022 at 09:25, Neil C Smith <ne...@apache.org> wrote:
>
> On Thu, 11 Aug 2022 at 14:20, Michael Bien <mb...@gmail.com> wrote:
> > > The branch tests run on the head of the delivery branch
> > > (refs/head/delivery). The sync PR tests run on the pending result of
> > > the merge to the base branch (refs/pull/<ID>/merge), eg. master.
>
> The above assumption might need checking. Just restarting a
> pull-request test on the sync PRs and noticed that it restarted in
> both - ie. two sync PRs with different base branches share a single
> set of tests on both GitHub and Travis. At the moment, it's the
> result of the merge to master, so the important one. Not sure how
> consistent the behaviour is - workflow history shows the other one
> cancelled.
OK, this is probably due to
https://github.com/apache/netbeans/pull/3932 I guess we need to group
by head and base ref?
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Neil C Smith <ne...@apache.org>.
On Thu, 11 Aug 2022 at 14:20, Michael Bien <mb...@gmail.com> wrote:
> > The branch tests run on the head of the delivery branch
> > (refs/head/delivery). The sync PR tests run on the pending result of
> > the merge to the base branch (refs/pull/<ID>/merge), eg. master.
The above assumption might need checking. Just restarting a
pull-request test on the sync PRs and noticed that it restarted in
both - ie. two sync PRs with different base branches share a single
set of tests on both GitHub and Travis. At the moment, it's the
result of the merge to master, so the important one. Not sure how
consistent the behaviour is - workflow history shows the other one
cancelled.
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 12.08.22 15:25, Eric Bresie wrote:
> Did I read correctly, it doesn’t run if there are no labels? What about
> new contributors without permissions to label yet for a given PR?
reviewers. PRs of new contributors won't get merged on first attempt
anyway, its an iterative process.
CI doesn't run for first time contributors and has to be unlocked by
committers for security/resource abuse reasons, this is already the case
today.
An unlabeled PR from a contributor would start CI but with many long
jobs/steps disabled. It was just a suggestion that maybe we should fail
the build job early if a PR is not labeled - this is not implemented.
> If the GitHub/travis practices are already documented somewhere then can
> they be updated (once accepted) to have a centralized place for knowledge
> of this (and the available labels)?
its documented in the yaml file, I am also planning to put a note on the
label description itself if it influences the build (using the colors
differently might be also an option).
https://github.com/apache/netbeans/pull/4431#discussion_r942356174
reviewers/project managers should take a look at this section. (I am esp
not sure about the JavaDoc label, since nobody gonna use it intuitively,
its still useful since javadoc is a fairly long step, we could add more
triggers for it, e.g "API Change" etc)
But in general: If this works correctly, you won't have to know about
it. All you have to do is to label the PR, it should "just work" since
the labels are not obscure flags. If your PR changes gradle, add
"Gradle", if your PR changes PHP, add "PHP". You should be doing this
anyway as committer and good github citizen so that reviewers don't have
to fix your PRs.
Reviewers should know about it though, since project wide cleanups would
for example require a ci:all-tests run before merge etc.
> If they are not then is it worth starting to so folks have a centralized
> place instead of searching through PRs or emails? Maybe somewhere near
> here (1)? Or maybe a new page for GitHub similar to page for Jenkins (2)
> with GitHub and GitHub Actions and practices documented?
having duplicated doc is dangerous and annoying (nobody wants to sync
docs if the impl changes), if you want it in a wiki, i would recommend
simply linking to the section in the yaml on github, maybe there is a
way to embed it.
best regards,
michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Eric Bresie <eb...@gmail.com>.
Did I read correctly, it doesn’t run if there are no labels? What about
new contributors without permissions to label yet for a given PR?
If the GitHub/travis practices are already documented somewhere then can
they be updated (once accepted) to have a centralized place for knowledge
of this (and the available labels)?
If they are not then is it worth starting to so folks have a centralized
place instead of searching through PRs or emails? Maybe somewhere near
here (1)? Or maybe a new page for GitHub similar to page for Jenkins (2)
with GitHub and GitHub Actions and practices documented?
References:
(1) https://cwiki.apache.org/confluence/x/wBJ4CQ
(2) https://cwiki.apache.org/confluence/x/VQ3HBg
On Thu, Aug 11, 2022 at 8:20 AM Michael Bien <mb...@gmail.com> wrote:
> On 11.08.22 11:21, Neil C Smith wrote:
> > On Thu, 11 Aug 2022 at 00:23, Michael Bien <mb...@gmail.com> wrote:
> >> Usually push only happens on merge (since that counts as push too), if
> >> you sync two branches on our main repo (not from clone to main), you
> >> would push into one branch and create a PR from it. Which means it
> >> builds the same hash twice just for the PR (and a third time on merge).
> >> You would see every check twice below the sync PR, one marked with push
> >> and one with pull_request. Unfortunately it doesn't show that on closed
> >> PRs and we don't have one open atm.
> > I'm aware of the two checks. My point was that they're not the same
> > hash, on GitHub or Travis, as far as I know.
> >
> > The branch tests run on the head of the delivery branch
> > (refs/head/delivery). The sync PR tests run on the pending result of
> > the merge to the base branch (refs/pull/<ID>/merge), eg. master. Now
> > the result of merging to releaseXXX is usually not interesting.
> > Testing on the result of merging to master can (and occasionally does)
> > highlight concerns. Which we can then try to address *before* merging
> > syncs to the two branches. PRs into delivery have usually been tested
> > against delivery, not master. This is a reason for opening the sync
> > PRs as soon as delivery has changes.
> >
> > This is somewhat of a tangent to your original post - just pointing
> > out that full CI on sync PRs is deliberate at the moment. We could
> > change procedures if you think there's a better way to achieve the
> > above.
>
> I see - makes sense. So assuming the proposed changes make it in,
> release managers would have to set the ci:all-tests label on the sync PR
> to have the same result as today.
>
> regards,
>
> michael
>
> >
> > Best wishes,
> >
> > Neil
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > For additional commands, e-mail: dev-help@netbeans.apache.org
> >
> > For further information about the NetBeans mailing lists, visit:
> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
> --
Eric Bresie
ebresie@gmail.com
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 11.08.22 11:21, Neil C Smith wrote:
> On Thu, 11 Aug 2022 at 00:23, Michael Bien <mb...@gmail.com> wrote:
>> Usually push only happens on merge (since that counts as push too), if
>> you sync two branches on our main repo (not from clone to main), you
>> would push into one branch and create a PR from it. Which means it
>> builds the same hash twice just for the PR (and a third time on merge).
>> You would see every check twice below the sync PR, one marked with push
>> and one with pull_request. Unfortunately it doesn't show that on closed
>> PRs and we don't have one open atm.
> I'm aware of the two checks. My point was that they're not the same
> hash, on GitHub or Travis, as far as I know.
>
> The branch tests run on the head of the delivery branch
> (refs/head/delivery). The sync PR tests run on the pending result of
> the merge to the base branch (refs/pull/<ID>/merge), eg. master. Now
> the result of merging to releaseXXX is usually not interesting.
> Testing on the result of merging to master can (and occasionally does)
> highlight concerns. Which we can then try to address *before* merging
> syncs to the two branches. PRs into delivery have usually been tested
> against delivery, not master. This is a reason for opening the sync
> PRs as soon as delivery has changes.
>
> This is somewhat of a tangent to your original post - just pointing
> out that full CI on sync PRs is deliberate at the moment. We could
> change procedures if you think there's a better way to achieve the
> above.
I see - makes sense. So assuming the proposed changes make it in,
release managers would have to set the ci:all-tests label on the sync PR
to have the same result as today.
regards,
michael
>
> Best wishes,
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Neil C Smith <ne...@apache.org>.
On Thu, 11 Aug 2022 at 00:23, Michael Bien <mb...@gmail.com> wrote:
> Usually push only happens on merge (since that counts as push too), if
> you sync two branches on our main repo (not from clone to main), you
> would push into one branch and create a PR from it. Which means it
> builds the same hash twice just for the PR (and a third time on merge).
> You would see every check twice below the sync PR, one marked with push
> and one with pull_request. Unfortunately it doesn't show that on closed
> PRs and we don't have one open atm.
I'm aware of the two checks. My point was that they're not the same
hash, on GitHub or Travis, as far as I know.
The branch tests run on the head of the delivery branch
(refs/head/delivery). The sync PR tests run on the pending result of
the merge to the base branch (refs/pull/<ID>/merge), eg. master. Now
the result of merging to releaseXXX is usually not interesting.
Testing on the result of merging to master can (and occasionally does)
highlight concerns. Which we can then try to address *before* merging
syncs to the two branches. PRs into delivery have usually been tested
against delivery, not master. This is a reason for opening the sync
PRs as soon as delivery has changes.
This is somewhat of a tangent to your original post - just pointing
out that full CI on sync PRs is deliberate at the moment. We could
change procedures if you think there's a better way to achieve the
above.
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
On 11.08.22 00:18, Neil C Smith wrote:
> On Wed, 10 Aug 2022 at 18:25, Michael Bien <mb...@gmail.com> wrote:
>> There are also additional command labels available, ci:all-tests would
>> enable everything, while ci:no-build would as you would guess disable
>> everything (e.g if all what the PR does is to edit a readme or sync-PRs
>> which syncs two branches could use this too, since they build everything
>> twice right now).
> Really like these changes overall. However, I'm not sure what this
> bit means - one of the reasons we use sync PRs is to run all CI tests
> on them.
I haven't explained that very well sorry.
main.yaml our main workflow triggers on pull_request and push to the
main repo.
Usually push only happens on merge (since that counts as push too), if
you sync two branches on our main repo (not from clone to main), you
would push into one branch and create a PR from it. Which means it
builds the same hash twice just for the PR (and a third time on merge).
You would see every check twice below the sync PR, one marked with push
and one with pull_request. Unfortunately it doesn't show that on closed
PRs and we don't have one open atm.
So if you would set a sync PR to ci:no-build, it would ignore the PR
trigger but still build the push trigger -> it should hopefully only
build once. But sync PRs happen so rarely that it probably isn't worth
to optimize for that scenario. Setting it to ci:all-tests should be fine ;).
regards,
michael
>
> Best wishes.
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Neil C Smith <ne...@apache.org>.
On Wed, 10 Aug 2022 at 18:25, Michael Bien <mb...@gmail.com> wrote:
> There are also additional command labels available, ci:all-tests would
> enable everything, while ci:no-build would as you would guess disable
> everything (e.g if all what the PR does is to edit a readme or sync-PRs
> which syncs two branches could use this too, since they build everything
> twice right now).
Really like these changes overall. However, I'm not sure what this
bit means - one of the reasons we use sync PRs is to run all CI tests
on them.
Best wishes.
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
some stats:
here an example of a 3h run (unlabeled PR):
https://github.com/apache/netbeans/actions/runs/2832175091/usage
running everything (ci:all-tests label or merge run) takes about 8.2h:
https://github.com/apache/netbeans/actions/runs/2899043480/usage
a typical run right now prior to the proposed PR is about 4h:
https://github.com/apache/netbeans/actions/runs/2894573402/usage
the job count increased from 12 (fixed) to 25 (conditional) while travis
time went down from 12.5h -> 5h
list of labels which influence CI:
https://github.com/apache/netbeans/pull/4431#pullrequestreview-1079689926
regards,
-mbien
On 10.08.22 19:22, Michael Bien wrote:
> Hello devs,
>
> while migrating some jobs from travis to github CI, I started making
> some (time consuming) jobs and/or build steps conditional based on PR
> labels.
>
> I keep it here short since most of the info is in the PR text, but as
> example:
>
> The new test job called "Build Tools" would only run if the PR is
> labeled with Ant, Gradle, Maven or MX.
>
> There are also additional command labels available, ci:all-tests would
> enable everything, while ci:no-build would as you would guess disable
> everything (e.g if all what the PR does is to edit a readme or
> sync-PRs which syncs two branches could use this too, since they build
> everything twice right now).
>
> This is just about PRs, the merge will always test everything on master.
>
>
> Why labels and not path based dependency checks? Because they never
> work outside of fairly trivial cases in my experience. They either
> test too much or not enough and whats worse: the dev has to second
> guess if the right things are actually being tested - if they aren't
> there must be also a second mechanism to override the automation.
>
> With labels, reviewers can decide what needs to be tested based on
> experience. A correctly labeled PR should "just work" most of the time
> even if you wouldn't know about the details. However, everyone still
> has to know about it: If you want the right things to be tested, you
> have to label the PR _before_ pressing the create button, otherwise
> the first run will have the wrong settings. (syncs will pick up new
> labels)
>
> I was considering to fail the build for PRs without labels (not
> implemented).
>
> https://github.com/apache/netbeans/pull/4431
>
> feedback appreciated,
>
> michael
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: running CI jobs based on PR labels
Posted by Michael Bien <mb...@gmail.com>.
should we proceed with this?
most important points:
- labeling PRs will be important, if you don't label anything: only
~3h of tests/builds will run
- labels are picked up before every CI run (changing labels and
syncing is fine)
- label your PR before pressing the create PR button, otherwise the
first run is wasted
- reviewers should check if sufficient testing was done before
merging, esp if its from new contributors
why all the effort?
- it should use far less resources in average
- more jobs on gh actions means we get better error reports if
something fails etc
- less travis
-> https://github.com/apache/netbeans/pull/4431
regards,
-mbien
On 10.08.22 19:22, Michael Bien wrote:
> Hello devs,
>
> while migrating some jobs from travis to github CI, I started making
> some (time consuming) jobs and/or build steps conditional based on PR
> labels.
>
> I keep it here short since most of the info is in the PR text, but as
> example:
>
> The new test job called "Build Tools" would only run if the PR is
> labeled with Ant, Gradle, Maven or MX.
>
> There are also additional command labels available, ci:all-tests would
> enable everything, while ci:no-build would as you would guess disable
> everything (e.g if all what the PR does is to edit a readme or
> sync-PRs which syncs two branches could use this too, since they build
> everything twice right now).
>
> This is just about PRs, the merge will always test everything on master.
>
>
> Why labels and not path based dependency checks? Because they never
> work outside of fairly trivial cases in my experience. They either
> test too much or not enough and whats worse: the dev has to second
> guess if the right things are actually being tested - if they aren't
> there must be also a second mechanism to override the automation.
>
> With labels, reviewers can decide what needs to be tested based on
> experience. A correctly labeled PR should "just work" most of the time
> even if you wouldn't know about the details. However, everyone still
> has to know about it: If you want the right things to be tested, you
> have to label the PR _before_ pressing the create button, otherwise
> the first run will have the wrong settings. (syncs will pick up new
> labels)
>
> I was considering to fail the build for PRs without labels (not
> implemented).
>
> https://github.com/apache/netbeans/pull/4431
>
> feedback appreciated,
>
> michael
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists