You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Chaitanya Bapat <ch...@gmail.com> on 2020/10/14 22:14:44 UTC
Re: [CI] Proposal for automatic PR labeling
Update: PR Status Label Bot is functional.
You'll see your PRs getting labels such as
- pr-awaiting-testing [When CI is kick-started for every commit]
- pr-work-in-progress [When CI has failed or PR is marked draft or WIP in
title]
- pr-pending-review [When CI succeeded & no review]
- pr-pending-merge [When CI succeeds & 2 Apache MXNet Committers approve]
- pr-awaiting-response [When CI succeeded but reviewer request changes]
PR: https://github.com/apache/incubator-mxnet-ci/pull/27
Followup PR to enable add/remove label :
https://github.com/apache/incubator-mxnet-ci/pull/33
On Sun, 2 Aug 2020 at 13:56, Chaitanya Bapat <ch...@gmail.com> wrote:
> Yes. It has been disabled.
>
> On Sun, 2 Aug 2020 at 13:14, Marco de Abreu <ma...@gmail.com>
> wrote:
>
>> Great. As I said, please make sure that the bot is disabled until it has
>> been reviewed and approved.
>>
>> -Marco
>>
>> On Sun, Aug 2, 2020, 10:12 PM Chaitanya Bapat <ch...@gmail.com>
>> wrote:
>>
>> > Thanks. webhook is setup!
>> >
>> > On Sun, 2 Aug 2020 at 02:43, Marco de Abreu <ma...@gmail.com>
>> > wrote:
>> >
>> > > Ticket created: https://issues.apache.org/jira/browse/INFRA-20624
>> > >
>> > > -Marco
>> > >
>> > > On Sun, Aug 2, 2020, 7:43 AM Chaitanya Bapat <ch...@gmail.com>
>> > wrote:
>> > >
>> > > > Right.
>> > > > I've sent you the email on your personal email address.
>> > > > Thanks for helping setup the github webhook with Apache Infra.
>> > > >
>> > > > Cheers
>> > > > Chai
>> > > >
>> > > > On Sat, 1 Aug 2020 at 03:25, Marco de Abreu <
>> marco.g.abreu@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Maybe we should just remove the pr-awaiting-testing state
>> entirely.
>> > > > Either
>> > > > > it's WIP or CI passed.
>> > > > >
>> > > > > -Marco
>> > > > >
>> > > > > On Sat, Aug 1, 2020 at 12:21 PM Marco de Abreu <
>> > > marco.g.abreu@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I generally like the idea, thanks for improving the user
>> experience
>> > > for
>> > > > > > the project, Chai! One thing that would be important to me is
>> > > avoiding
>> > > > a
>> > > > > > constant shifting between the states. The PR shouldn't switch to
>> > > > > > "awaiting-testing" every single time CI is retriggered but
>> rather
>> > > have
>> > > > > some
>> > > > > > consistency. But that's something we can also tweak as we go.
>> > > > > >
>> > > > > > Before the bot is run, a small heads up would be nice. Just
>> > running a
>> > > > > > batch job on the repository - especially using others'
>> credentials
>> > -
>> > > > > > without previous approval is not that nice. Hence, please run
>> your
>> > > > tests
>> > > > > a
>> > > > > > non-intrusive way until the bot has been approved for
>> deployment.
>> > > > > >
>> > > > > > With regards to the ticket, I'm happy to be of assistance. Just
>> > send
>> > > > the
>> > > > > > secret to my personal email.
>> > > > > >
>> > > > > > Best regards
>> > > > > > Marco
>> > > > > >
>> > > > > > On Fri, Jul 31, 2020 at 9:12 PM Sheng Zha <zh...@apache.org>
>> > > wrote:
>> > > > > >
>> > > > > >> +1, thanks for driving this. Here's the pseudo code for the PR
>> > > > labeling
>> > > > > >> for better readability:
>> > > > > >>
>> > > > > >> if WIP in title or PR is draft or CI failed:
>> > > > > >> pr-work-in-progress
>> > > > > >> elif CI has not started yet or CI is in progress:
>> > > > > >> pr-awaiting-testing
>> > > > > >> else: # CI passed checks
>> > > > > >> if pr has at least one approval and no request changes:
>> > > > > >> pr-awaiting-merge
>> > > > > >> elif pr has no review or all reviews have been
>> > > > > dismissed/re-requested:
>> > > > > >> pr-awaiting-review
>> > > > > >> else: # pr has a review that hasn't been dismissed yet no
>> > > approval
>> > > > > >> pr-awaiting-response
>> > > > > >>
>> > > > > >> Best,
>> > > > > >> Sheng
>> > > > > >>
>> > > > > >> On 2020/07/31 18:08:34, Chaitanya Bapat <ch...@gmail.com>
>> > > wrote:
>> > > > > >> > Hello MXNet community,
>> > > > > >> >
>> > > > > >> > Banking on the success of mxnet-bot that has been triggering
>> CI
>> > > > > >> > successfully for the past 3 months, Sheng proposed an idea
>> for
>> > > > > >> automatic PR
>> > > > > >> > labeling. I am working on the implementation of the same.
>> > > > > >> >
>> > > > > >> > *What's the idea?*
>> > > > > >> > An automated way of labeling PRs based on the status of CI.
>> > > > > >> >
>> > > > > >> > *Why?*
>> > > > > >> >
>> > > > > >> > This serves dual purpose
>> > > > > >> >
>> > > > > >> > - Automates the labeling [currently manual]
>> > > > > >> > - Allows contributors to prioritize review of PRs.
>> > > > > >> >
>> > > > > >> > *How?*
>> > > > > >> > 1. Github webhook would send status update events to the
>> Lambda
>> > > > > function
>> > > > > >> > 2. Lambda would then label the CI based on following
>> conditions
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > *if WIP in title or PR is draft or CI failed:
>> > > > > >> pr-work-in-progresselif CI
>> > > > > >> > has not started yet or CI is in progress:
>> > > > pr-awaiting-testingelse:
>> > > > > #
>> > > > > >> CI
>> > > > > >> > passed checks if pr has at least one approval and no
>> request
>> > > > > changes:
>> > > > > >> > pr-awaiting-merge elif pr has no review or all reviews
>> > have
>> > > > > been
>> > > > > >> > dismissed/re-requested: pr-awaiting-review else: #
>> pr
>> > > has
>> > > > a
>> > > > > >> > review that hasn't been dismissed yet no approval
>> > > > > >> > pr-awaiting-response*
>> > > > > >> >
>> > > > > >> > To that end, a POC was done to test if it's possible &
>> result:
>> > it
>> > > > > works.
>> > > > > >> > Results of 1 successful run on Prod for
>> apache/incubator-mxnet
>> > > repo
>> > > > > >> >
>> > > > > >> > Open PRs : 207
>> > > > > >> > PRs labeled : 18
>> > > > > >> > PRs already labeled : 38
>> > > > > >> > PRs with unknown jobs : 2
>> > > > > >> > PRs with status failed : 149
>> > > > > >> >
>> > > > > >> > Lambda function labeled 18 PRs that had passed all the tests.
>> > > > > >> >
>> > > > > >> > PR for POC:
>> > https://github.com/apache/incubator-mxnet-ci/pull/27
>> > > > > >> > However, Marco made a valid point regarding leveraging Github
>> > > > Webhooks
>> > > > > >> > [instead of Jenkins] for retrieving status of the PR.
>> > > > > >> >
>> > > > > >> > *What's required?*
>> > > > > >> > Thus, to get this feature functional, I'd need a separate
>> Github
>> > > > > webhook
>> > > > > >> > that sends information related to PR Status. Just like the
>> > webhook
>> > > > > >> request
>> > > > > >> > for CI Bot, I can send the details over private email to
>> Marco's
>> > > > > >> personal
>> > > > > >> > email address & then he can create a ticket with Apache
>> Infra.
>> > > > > >> >
>> > > > > >> > Thanks to Sheng & Marco for guidance & assistance on this.
>> > > > > >> >
>> > > > > >> > Thanks,
>> > > > > >> > Chai
>> > > > > >> >
>> > > > > >> > --
>> > > > > >> > *Chaitanya Prakash Bapat*
>> > > > > >> > *+1 (973) 953-6299*
>> > > > > >> >
>> > > > > >> > [image: https://www.linkedin.com//in/chaibapat25]
>> > > > > >> > <https://github.com/ChaiBapchya>[image:
>> > > > > >> https://www.facebook.com/chaibapat]
>> > > > > >> > <https://www.facebook.com/chaibapchya>[image:
>> > > > > >> > https://twitter.com/ChaiBapchya] <
>> > https://twitter.com/ChaiBapchya
>> > > > > >> >[image:
>> > > > > >> > https://www.linkedin.com//in/chaibapat25]
>> > > > > >> > <https://www.linkedin.com//in/chaibapchya/>
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Chaitanya Prakash Bapat*
>> > > > *+1 (973) 953-6299*
>> > > >
>> > > > [image: https://www.linkedin.com//in/chaibapat25]
>> > > > <https://github.com/ChaiBapchya>[image:
>> > > https://www.facebook.com/chaibapat
>> > > > ]
>> > > > <https://www.facebook.com/chaibapchya>[image:
>> > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
>> > > >[image:
>> > > > https://www.linkedin.com//in/chaibapat25]
>> > > > <https://www.linkedin.com//in/chaibapchya/>
>> > > >
>> > >
>> >
>> >
>> > --
>> > *Chaitanya Prakash Bapat*
>> > *+1 (973) 953-6299*
>> >
>> > [image: https://www.linkedin.com//in/chaibapat25]
>> > <https://github.com/ChaiBapchya>[image:
>> https://www.facebook.com/chaibapat
>> > ]
>> > <https://www.facebook.com/chaibapchya>[image:
>> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
>> >[image:
>> > https://www.linkedin.com//in/chaibapat25]
>> > <https://www.linkedin.com//in/chaibapchya/>
>> >
>>
>
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image:
> https://www.facebook.com/chaibapat] <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
>
--
*Chaitanya Prakash Bapat*
*+1 (973) 953-6299*
[image: https://www.linkedin.com//in/chaibapat25]
<https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat]
<https://www.facebook.com/chaibapchya>[image:
https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
https://www.linkedin.com//in/chaibapat25]
<https://www.linkedin.com//in/chaibapchya/>
Re: [CI] Proposal for automatic PR labeling
Posted by Chaitanya Bapat <ch...@gmail.com>.
Sure. Few others reported the clutter due to state switches. Also the
Github UI shows these label switches in same line confounding the users.
So maybe, reducing the states can help declutter.
I'd let others comment and wait till Saturday for developers to use it and
get a feel.
If no one opposes the change till Saturday, then I'll drop support for
pr-awaiting-testing label
On Wed, 14 Oct 2020 at 16:02, Marco de Abreu <ma...@gmail.com>
wrote:
> Thanks chai!
>
> Shall we remove the awaiting testing state? To me it doesn't really provide
> any value and will just produce constant state switches and clutter in the
> PR. I would say that the state should not change while CI is still running
> until it reached a conclusion - whether that's one failed or all passed.
>
> -Marco
>
>
> On Thu, Oct 15, 2020, 00:15 Chaitanya Bapat <ch...@gmail.com> wrote:
>
> > Update: PR Status Label Bot is functional.
> >
> > You'll see your PRs getting labels such as
> > - pr-awaiting-testing [When CI is kick-started for every commit]
> > - pr-work-in-progress [When CI has failed or PR is marked draft or WIP in
> > title]
> > - pr-pending-review [When CI succeeded & no review]
> > - pr-pending-merge [When CI succeeds & 2 Apache MXNet Committers approve]
> > - pr-awaiting-response [When CI succeeded but reviewer request changes]
> >
> > PR: https://github.com/apache/incubator-mxnet-ci/pull/27
> > Followup PR to enable add/remove label :
> > https://github.com/apache/incubator-mxnet-ci/pull/33
> >
> >
> > On Sun, 2 Aug 2020 at 13:56, Chaitanya Bapat <ch...@gmail.com>
> wrote:
> >
> > > Yes. It has been disabled.
> > >
> > > On Sun, 2 Aug 2020 at 13:14, Marco de Abreu <ma...@gmail.com>
> > > wrote:
> > >
> > >> Great. As I said, please make sure that the bot is disabled until it
> has
> > >> been reviewed and approved.
> > >>
> > >> -Marco
> > >>
> > >> On Sun, Aug 2, 2020, 10:12 PM Chaitanya Bapat <ch...@gmail.com>
> > >> wrote:
> > >>
> > >> > Thanks. webhook is setup!
> > >> >
> > >> > On Sun, 2 Aug 2020 at 02:43, Marco de Abreu <
> marco.g.abreu@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Ticket created: https://issues.apache.org/jira/browse/INFRA-20624
> > >> > >
> > >> > > -Marco
> > >> > >
> > >> > > On Sun, Aug 2, 2020, 7:43 AM Chaitanya Bapat <
> chai.bapat@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > > Right.
> > >> > > > I've sent you the email on your personal email address.
> > >> > > > Thanks for helping setup the github webhook with Apache Infra.
> > >> > > >
> > >> > > > Cheers
> > >> > > > Chai
> > >> > > >
> > >> > > > On Sat, 1 Aug 2020 at 03:25, Marco de Abreu <
> > >> marco.g.abreu@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Maybe we should just remove the pr-awaiting-testing state
> > >> entirely.
> > >> > > > Either
> > >> > > > > it's WIP or CI passed.
> > >> > > > >
> > >> > > > > -Marco
> > >> > > > >
> > >> > > > > On Sat, Aug 1, 2020 at 12:21 PM Marco de Abreu <
> > >> > > marco.g.abreu@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi,
> > >> > > > > >
> > >> > > > > > I generally like the idea, thanks for improving the user
> > >> experience
> > >> > > for
> > >> > > > > > the project, Chai! One thing that would be important to me
> is
> > >> > > avoiding
> > >> > > > a
> > >> > > > > > constant shifting between the states. The PR shouldn't
> switch
> > to
> > >> > > > > > "awaiting-testing" every single time CI is retriggered but
> > >> rather
> > >> > > have
> > >> > > > > some
> > >> > > > > > consistency. But that's something we can also tweak as we
> go.
> > >> > > > > >
> > >> > > > > > Before the bot is run, a small heads up would be nice. Just
> > >> > running a
> > >> > > > > > batch job on the repository - especially using others'
> > >> credentials
> > >> > -
> > >> > > > > > without previous approval is not that nice. Hence, please
> run
> > >> your
> > >> > > > tests
> > >> > > > > a
> > >> > > > > > non-intrusive way until the bot has been approved for
> > >> deployment.
> > >> > > > > >
> > >> > > > > > With regards to the ticket, I'm happy to be of assistance.
> > Just
> > >> > send
> > >> > > > the
> > >> > > > > > secret to my personal email.
> > >> > > > > >
> > >> > > > > > Best regards
> > >> > > > > > Marco
> > >> > > > > >
> > >> > > > > > On Fri, Jul 31, 2020 at 9:12 PM Sheng Zha <
> > zhasheng@apache.org>
> > >> > > wrote:
> > >> > > > > >
> > >> > > > > >> +1, thanks for driving this. Here's the pseudo code for the
> > PR
> > >> > > > labeling
> > >> > > > > >> for better readability:
> > >> > > > > >>
> > >> > > > > >> if WIP in title or PR is draft or CI failed:
> > >> > > > > >> pr-work-in-progress
> > >> > > > > >> elif CI has not started yet or CI is in progress:
> > >> > > > > >> pr-awaiting-testing
> > >> > > > > >> else: # CI passed checks
> > >> > > > > >> if pr has at least one approval and no request changes:
> > >> > > > > >> pr-awaiting-merge
> > >> > > > > >> elif pr has no review or all reviews have been
> > >> > > > > dismissed/re-requested:
> > >> > > > > >> pr-awaiting-review
> > >> > > > > >> else: # pr has a review that hasn't been dismissed yet
> no
> > >> > > approval
> > >> > > > > >> pr-awaiting-response
> > >> > > > > >>
> > >> > > > > >> Best,
> > >> > > > > >> Sheng
> > >> > > > > >>
> > >> > > > > >> On 2020/07/31 18:08:34, Chaitanya Bapat <
> > chai.bapat@gmail.com>
> > >> > > wrote:
> > >> > > > > >> > Hello MXNet community,
> > >> > > > > >> >
> > >> > > > > >> > Banking on the success of mxnet-bot that has been
> > triggering
> > >> CI
> > >> > > > > >> > successfully for the past 3 months, Sheng proposed an
> idea
> > >> for
> > >> > > > > >> automatic PR
> > >> > > > > >> > labeling. I am working on the implementation of the same.
> > >> > > > > >> >
> > >> > > > > >> > *What's the idea?*
> > >> > > > > >> > An automated way of labeling PRs based on the status of
> CI.
> > >> > > > > >> >
> > >> > > > > >> > *Why?*
> > >> > > > > >> >
> > >> > > > > >> > This serves dual purpose
> > >> > > > > >> >
> > >> > > > > >> > - Automates the labeling [currently manual]
> > >> > > > > >> > - Allows contributors to prioritize review of PRs.
> > >> > > > > >> >
> > >> > > > > >> > *How?*
> > >> > > > > >> > 1. Github webhook would send status update events to the
> > >> Lambda
> > >> > > > > function
> > >> > > > > >> > 2. Lambda would then label the CI based on following
> > >> conditions
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > *if WIP in title or PR is draft or CI failed:
> > >> > > > > >> pr-work-in-progresselif CI
> > >> > > > > >> > has not started yet or CI is in progress:
> > >> > > > pr-awaiting-testingelse:
> > >> > > > > #
> > >> > > > > >> CI
> > >> > > > > >> > passed checks if pr has at least one approval and no
> > >> request
> > >> > > > > changes:
> > >> > > > > >> > pr-awaiting-merge elif pr has no review or all
> > reviews
> > >> > have
> > >> > > > > been
> > >> > > > > >> > dismissed/re-requested: pr-awaiting-review
> else:
> > #
> > >> pr
> > >> > > has
> > >> > > > a
> > >> > > > > >> > review that hasn't been dismissed yet no approval
> > >> > > > > >> > pr-awaiting-response*
> > >> > > > > >> >
> > >> > > > > >> > To that end, a POC was done to test if it's possible &
> > >> result:
> > >> > it
> > >> > > > > works.
> > >> > > > > >> > Results of 1 successful run on Prod for
> > >> apache/incubator-mxnet
> > >> > > repo
> > >> > > > > >> >
> > >> > > > > >> > Open PRs : 207
> > >> > > > > >> > PRs labeled : 18
> > >> > > > > >> > PRs already labeled : 38
> > >> > > > > >> > PRs with unknown jobs : 2
> > >> > > > > >> > PRs with status failed : 149
> > >> > > > > >> >
> > >> > > > > >> > Lambda function labeled 18 PRs that had passed all the
> > tests.
> > >> > > > > >> >
> > >> > > > > >> > PR for POC:
> > >> > https://github.com/apache/incubator-mxnet-ci/pull/27
> > >> > > > > >> > However, Marco made a valid point regarding leveraging
> > Github
> > >> > > > Webhooks
> > >> > > > > >> > [instead of Jenkins] for retrieving status of the PR.
> > >> > > > > >> >
> > >> > > > > >> > *What's required?*
> > >> > > > > >> > Thus, to get this feature functional, I'd need a separate
> > >> Github
> > >> > > > > webhook
> > >> > > > > >> > that sends information related to PR Status. Just like
> the
> > >> > webhook
> > >> > > > > >> request
> > >> > > > > >> > for CI Bot, I can send the details over private email to
> > >> Marco's
> > >> > > > > >> personal
> > >> > > > > >> > email address & then he can create a ticket with Apache
> > >> Infra.
> > >> > > > > >> >
> > >> > > > > >> > Thanks to Sheng & Marco for guidance & assistance on
> this.
> > >> > > > > >> >
> > >> > > > > >> > Thanks,
> > >> > > > > >> > Chai
> > >> > > > > >> >
> > >> > > > > >> > --
> > >> > > > > >> > *Chaitanya Prakash Bapat*
> > >> > > > > >> > *+1 (973) 953-6299*
> > >> > > > > >> >
> > >> > > > > >> > [image: https://www.linkedin.com//in/chaibapat25]
> > >> > > > > >> > <https://github.com/ChaiBapchya>[image:
> > >> > > > > >> https://www.facebook.com/chaibapat]
> > >> > > > > >> > <https://www.facebook.com/chaibapchya>[image:
> > >> > > > > >> > https://twitter.com/ChaiBapchya] <
> > >> > https://twitter.com/ChaiBapchya
> > >> > > > > >> >[image:
> > >> > > > > >> > https://www.linkedin.com//in/chaibapat25]
> > >> > > > > >> > <https://www.linkedin.com//in/chaibapchya/>
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > *Chaitanya Prakash Bapat*
> > >> > > > *+1 (973) 953-6299*
> > >> > > >
> > >> > > > [image: https://www.linkedin.com//in/chaibapat25]
> > >> > > > <https://github.com/ChaiBapchya>[image:
> > >> > > https://www.facebook.com/chaibapat
> > >> > > > ]
> > >> > > > <https://www.facebook.com/chaibapchya>[image:
> > >> > > > https://twitter.com/ChaiBapchya] <
> https://twitter.com/ChaiBapchya
> > >> > > >[image:
> > >> > > > https://www.linkedin.com//in/chaibapat25]
> > >> > > > <https://www.linkedin.com//in/chaibapchya/>
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> > --
> > >> > *Chaitanya Prakash Bapat*
> > >> > *+1 (973) 953-6299*
> > >> >
> > >> > [image: https://www.linkedin.com//in/chaibapat25]
> > >> > <https://github.com/ChaiBapchya>[image:
> > >> https://www.facebook.com/chaibapat
> > >> > ]
> > >> > <https://www.facebook.com/chaibapchya>[image:
> > >> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> > >> >[image:
> > >> > https://www.linkedin.com//in/chaibapat25]
> > >> > <https://www.linkedin.com//in/chaibapchya/>
> > >> >
> > >>
> > >
> > >
> > > --
> > > *Chaitanya Prakash Bapat*
> > > *+1 (973) 953-6299*
> > >
> > > [image: https://www.linkedin.com//in/chaibapat25]
> > > <https://github.com/ChaiBapchya>[image:
> > > https://www.facebook.com/chaibapat] <
> > https://www.facebook.com/chaibapchya>[image:
> > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> > >[image:
> > > https://www.linkedin.com//in/chaibapat25]
> > > <https://www.linkedin.com//in/chaibapchya/>
> > >
> >
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > <https://github.com/ChaiBapchya>[image:
> https://www.facebook.com/chaibapat
> > ]
> > <https://www.facebook.com/chaibapchya>[image:
> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >[image:
> > https://www.linkedin.com//in/chaibapat25]
> > <https://www.linkedin.com//in/chaibapchya/>
> >
>
--
*Chaitanya Prakash Bapat*
*+1 (973) 953-6299*
[image: https://www.linkedin.com//in/chaibapat25]
<https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat]
<https://www.facebook.com/chaibapchya>[image:
https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
https://www.linkedin.com//in/chaibapat25]
<https://www.linkedin.com//in/chaibapchya/>
Re: [CI] Proposal for automatic PR labeling
Posted by Marco de Abreu <ma...@gmail.com>.
Thanks chai!
Shall we remove the awaiting testing state? To me it doesn't really provide
any value and will just produce constant state switches and clutter in the
PR. I would say that the state should not change while CI is still running
until it reached a conclusion - whether that's one failed or all passed.
-Marco
On Thu, Oct 15, 2020, 00:15 Chaitanya Bapat <ch...@gmail.com> wrote:
> Update: PR Status Label Bot is functional.
>
> You'll see your PRs getting labels such as
> - pr-awaiting-testing [When CI is kick-started for every commit]
> - pr-work-in-progress [When CI has failed or PR is marked draft or WIP in
> title]
> - pr-pending-review [When CI succeeded & no review]
> - pr-pending-merge [When CI succeeds & 2 Apache MXNet Committers approve]
> - pr-awaiting-response [When CI succeeded but reviewer request changes]
>
> PR: https://github.com/apache/incubator-mxnet-ci/pull/27
> Followup PR to enable add/remove label :
> https://github.com/apache/incubator-mxnet-ci/pull/33
>
>
> On Sun, 2 Aug 2020 at 13:56, Chaitanya Bapat <ch...@gmail.com> wrote:
>
> > Yes. It has been disabled.
> >
> > On Sun, 2 Aug 2020 at 13:14, Marco de Abreu <ma...@gmail.com>
> > wrote:
> >
> >> Great. As I said, please make sure that the bot is disabled until it has
> >> been reviewed and approved.
> >>
> >> -Marco
> >>
> >> On Sun, Aug 2, 2020, 10:12 PM Chaitanya Bapat <ch...@gmail.com>
> >> wrote:
> >>
> >> > Thanks. webhook is setup!
> >> >
> >> > On Sun, 2 Aug 2020 at 02:43, Marco de Abreu <ma...@gmail.com>
> >> > wrote:
> >> >
> >> > > Ticket created: https://issues.apache.org/jira/browse/INFRA-20624
> >> > >
> >> > > -Marco
> >> > >
> >> > > On Sun, Aug 2, 2020, 7:43 AM Chaitanya Bapat <ch...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Right.
> >> > > > I've sent you the email on your personal email address.
> >> > > > Thanks for helping setup the github webhook with Apache Infra.
> >> > > >
> >> > > > Cheers
> >> > > > Chai
> >> > > >
> >> > > > On Sat, 1 Aug 2020 at 03:25, Marco de Abreu <
> >> marco.g.abreu@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Maybe we should just remove the pr-awaiting-testing state
> >> entirely.
> >> > > > Either
> >> > > > > it's WIP or CI passed.
> >> > > > >
> >> > > > > -Marco
> >> > > > >
> >> > > > > On Sat, Aug 1, 2020 at 12:21 PM Marco de Abreu <
> >> > > marco.g.abreu@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I generally like the idea, thanks for improving the user
> >> experience
> >> > > for
> >> > > > > > the project, Chai! One thing that would be important to me is
> >> > > avoiding
> >> > > > a
> >> > > > > > constant shifting between the states. The PR shouldn't switch
> to
> >> > > > > > "awaiting-testing" every single time CI is retriggered but
> >> rather
> >> > > have
> >> > > > > some
> >> > > > > > consistency. But that's something we can also tweak as we go.
> >> > > > > >
> >> > > > > > Before the bot is run, a small heads up would be nice. Just
> >> > running a
> >> > > > > > batch job on the repository - especially using others'
> >> credentials
> >> > -
> >> > > > > > without previous approval is not that nice. Hence, please run
> >> your
> >> > > > tests
> >> > > > > a
> >> > > > > > non-intrusive way until the bot has been approved for
> >> deployment.
> >> > > > > >
> >> > > > > > With regards to the ticket, I'm happy to be of assistance.
> Just
> >> > send
> >> > > > the
> >> > > > > > secret to my personal email.
> >> > > > > >
> >> > > > > > Best regards
> >> > > > > > Marco
> >> > > > > >
> >> > > > > > On Fri, Jul 31, 2020 at 9:12 PM Sheng Zha <
> zhasheng@apache.org>
> >> > > wrote:
> >> > > > > >
> >> > > > > >> +1, thanks for driving this. Here's the pseudo code for the
> PR
> >> > > > labeling
> >> > > > > >> for better readability:
> >> > > > > >>
> >> > > > > >> if WIP in title or PR is draft or CI failed:
> >> > > > > >> pr-work-in-progress
> >> > > > > >> elif CI has not started yet or CI is in progress:
> >> > > > > >> pr-awaiting-testing
> >> > > > > >> else: # CI passed checks
> >> > > > > >> if pr has at least one approval and no request changes:
> >> > > > > >> pr-awaiting-merge
> >> > > > > >> elif pr has no review or all reviews have been
> >> > > > > dismissed/re-requested:
> >> > > > > >> pr-awaiting-review
> >> > > > > >> else: # pr has a review that hasn't been dismissed yet no
> >> > > approval
> >> > > > > >> pr-awaiting-response
> >> > > > > >>
> >> > > > > >> Best,
> >> > > > > >> Sheng
> >> > > > > >>
> >> > > > > >> On 2020/07/31 18:08:34, Chaitanya Bapat <
> chai.bapat@gmail.com>
> >> > > wrote:
> >> > > > > >> > Hello MXNet community,
> >> > > > > >> >
> >> > > > > >> > Banking on the success of mxnet-bot that has been
> triggering
> >> CI
> >> > > > > >> > successfully for the past 3 months, Sheng proposed an idea
> >> for
> >> > > > > >> automatic PR
> >> > > > > >> > labeling. I am working on the implementation of the same.
> >> > > > > >> >
> >> > > > > >> > *What's the idea?*
> >> > > > > >> > An automated way of labeling PRs based on the status of CI.
> >> > > > > >> >
> >> > > > > >> > *Why?*
> >> > > > > >> >
> >> > > > > >> > This serves dual purpose
> >> > > > > >> >
> >> > > > > >> > - Automates the labeling [currently manual]
> >> > > > > >> > - Allows contributors to prioritize review of PRs.
> >> > > > > >> >
> >> > > > > >> > *How?*
> >> > > > > >> > 1. Github webhook would send status update events to the
> >> Lambda
> >> > > > > function
> >> > > > > >> > 2. Lambda would then label the CI based on following
> >> conditions
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > *if WIP in title or PR is draft or CI failed:
> >> > > > > >> pr-work-in-progresselif CI
> >> > > > > >> > has not started yet or CI is in progress:
> >> > > > pr-awaiting-testingelse:
> >> > > > > #
> >> > > > > >> CI
> >> > > > > >> > passed checks if pr has at least one approval and no
> >> request
> >> > > > > changes:
> >> > > > > >> > pr-awaiting-merge elif pr has no review or all
> reviews
> >> > have
> >> > > > > been
> >> > > > > >> > dismissed/re-requested: pr-awaiting-review else:
> #
> >> pr
> >> > > has
> >> > > > a
> >> > > > > >> > review that hasn't been dismissed yet no approval
> >> > > > > >> > pr-awaiting-response*
> >> > > > > >> >
> >> > > > > >> > To that end, a POC was done to test if it's possible &
> >> result:
> >> > it
> >> > > > > works.
> >> > > > > >> > Results of 1 successful run on Prod for
> >> apache/incubator-mxnet
> >> > > repo
> >> > > > > >> >
> >> > > > > >> > Open PRs : 207
> >> > > > > >> > PRs labeled : 18
> >> > > > > >> > PRs already labeled : 38
> >> > > > > >> > PRs with unknown jobs : 2
> >> > > > > >> > PRs with status failed : 149
> >> > > > > >> >
> >> > > > > >> > Lambda function labeled 18 PRs that had passed all the
> tests.
> >> > > > > >> >
> >> > > > > >> > PR for POC:
> >> > https://github.com/apache/incubator-mxnet-ci/pull/27
> >> > > > > >> > However, Marco made a valid point regarding leveraging
> Github
> >> > > > Webhooks
> >> > > > > >> > [instead of Jenkins] for retrieving status of the PR.
> >> > > > > >> >
> >> > > > > >> > *What's required?*
> >> > > > > >> > Thus, to get this feature functional, I'd need a separate
> >> Github
> >> > > > > webhook
> >> > > > > >> > that sends information related to PR Status. Just like the
> >> > webhook
> >> > > > > >> request
> >> > > > > >> > for CI Bot, I can send the details over private email to
> >> Marco's
> >> > > > > >> personal
> >> > > > > >> > email address & then he can create a ticket with Apache
> >> Infra.
> >> > > > > >> >
> >> > > > > >> > Thanks to Sheng & Marco for guidance & assistance on this.
> >> > > > > >> >
> >> > > > > >> > Thanks,
> >> > > > > >> > Chai
> >> > > > > >> >
> >> > > > > >> > --
> >> > > > > >> > *Chaitanya Prakash Bapat*
> >> > > > > >> > *+1 (973) 953-6299*
> >> > > > > >> >
> >> > > > > >> > [image: https://www.linkedin.com//in/chaibapat25]
> >> > > > > >> > <https://github.com/ChaiBapchya>[image:
> >> > > > > >> https://www.facebook.com/chaibapat]
> >> > > > > >> > <https://www.facebook.com/chaibapchya>[image:
> >> > > > > >> > https://twitter.com/ChaiBapchya] <
> >> > https://twitter.com/ChaiBapchya
> >> > > > > >> >[image:
> >> > > > > >> > https://www.linkedin.com//in/chaibapat25]
> >> > > > > >> > <https://www.linkedin.com//in/chaibapchya/>
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > *Chaitanya Prakash Bapat*
> >> > > > *+1 (973) 953-6299*
> >> > > >
> >> > > > [image: https://www.linkedin.com//in/chaibapat25]
> >> > > > <https://github.com/ChaiBapchya>[image:
> >> > > https://www.facebook.com/chaibapat
> >> > > > ]
> >> > > > <https://www.facebook.com/chaibapchya>[image:
> >> > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >> > > >[image:
> >> > > > https://www.linkedin.com//in/chaibapat25]
> >> > > > <https://www.linkedin.com//in/chaibapchya/>
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > *Chaitanya Prakash Bapat*
> >> > *+1 (973) 953-6299*
> >> >
> >> > [image: https://www.linkedin.com//in/chaibapat25]
> >> > <https://github.com/ChaiBapchya>[image:
> >> https://www.facebook.com/chaibapat
> >> > ]
> >> > <https://www.facebook.com/chaibapchya>[image:
> >> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >> >[image:
> >> > https://www.linkedin.com//in/chaibapat25]
> >> > <https://www.linkedin.com//in/chaibapchya/>
> >> >
> >>
> >
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > <https://github.com/ChaiBapchya>[image:
> > https://www.facebook.com/chaibapat] <
> https://www.facebook.com/chaibapchya>[image:
> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >[image:
> > https://www.linkedin.com//in/chaibapat25]
> > <https://www.linkedin.com//in/chaibapchya/>
> >
>
>
> --
> *Chaitanya Prakash Bapat*
> *+1 (973) 953-6299*
>
> [image: https://www.linkedin.com//in/chaibapat25]
> <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat
> ]
> <https://www.facebook.com/chaibapchya>[image:
> https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image:
> https://www.linkedin.com//in/chaibapat25]
> <https://www.linkedin.com//in/chaibapchya/>
>