[DISCUSS] PR automation workflow

[Raúl Cumplido <ra...@gmail.com>]

Hi,

I would like to start working on some automation for our PRs and issues
workflows.

I've heard, and have experienced, the frustration of spending a lot of time
on our issue tracker and our PRs to follow up on their status.
It is very hard to keep track of which PRs and issues are waiting for user
feedback, have gone stale or are pending maintainer/committer action.
This means users frequently get no timely response, all the while we
regularly spend time on GH to look for PRs / issues needing action from us.
As a first step we should probably tackle PRs, once PRs are tackled and we
are satisfied with the solution, we can try to devise a similar one for GH
issues.

An example of a great improvement is the CODEOWNERS addition [1]. This
allows us to use filters like `is:pr is:open user-review-requested:@me` [2]
which will show PRs that have requested a review from us. This does not
solve the problem of what are the PRs waiting for second review,
waiting for changes, etcetera.

I don't think we have to reinvent the wheel, CPython has something that
works well and can easily be adapted/tweaked.
They use a GitHub bot (bedevere) with the following state machine:
https://github.com/python/bedevere#pr-state-machine

PRs have one label of the following workflow labels, depending of the state:
- `Awaiting review`
- `Awaiting core review`
- `Awaiting changes`
- `Awaiting change review`
- `Awaiting merge`

I would like to propose adding a GitHub bot to our repo that triggers on PR
changes / comments implementing a similar workflow than the one on the
CPython repository.

I am going to start working on it and I would love to hear feedback about
that workflow. I have also created an issue on the Repo [3].

Kind regards,
Raúl

[1] https://github.com/apache/arrow/pull/33622
[2]
https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+user-review-requested%3A%40me+
[3] https://github.com/apache/arrow/issues/33977

Re: [DISCUSS] PR automation workflow

[Rok Mihevc <ro...@gmail.com>]

+1 to Nic's comment.

On Fri, Feb 3, 2023 at 12:46 PM Nic Crane <th...@gmail.com> wrote:

> I have no specific comments on the what/how, other than to say I'm strongly
> in favour of some kind of system being implemented and tried out, as I
> currently rely on manual processes that are inefficient and make it easy to
> miss PRs which need looking at.
>
> On Thu, 2 Feb 2023 at 18:16, Andrew Lamb <al...@influxdata.com> wrote:
>
> > A process that we use in arrow-rs / arrow-datafusion,  which is less
> > precise but seems to be working well enough at the moment, is :
> >
> > 1. Mark  PRs that have received feedback and need more work prior to
> merge
> > from `Ready to Review` back to `Draft`
> > 2. Ask the author to set it back to "ready to review" when it is ready
> for
> > the next round of review
> >
> > Andrew
> >
> >
> > On Thu, Feb 2, 2023 at 4:17 AM Antoine Pitrou <an...@python.org>
> wrote:
> >
> > >
> > > Hi Raul,
> > >
> > > Since I'm the one who proposed that we reuse CPython's existing
> workflow
> > > infrastructure, it follows logically that I'm in favour :-)
> > >
> > > I'm a CPython core developer myself (though inactive lately), I will
> add
> > > that this workflow is really easing the work of reviewing PRs, as it
> > > makes obvious whether a PR is needing attention from a committer.
> > >
> > > Once we start working with it, we may decide to make adjustments to
> > > better fit our expectations, but I think we can start with the
> > > unmodified workflow scheme.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > > Le 01/02/2023 à 15:34, Raúl Cumplido a écrit :
> > > > Hi,
> > > >
> > > > I would like to start working on some automation for our PRs and
> issues
> > > > workflows.
> > > >
> > > > I've heard, and have experienced, the frustration of spending a lot
> of
> > > time
> > > > on our issue tracker and our PRs to follow up on their status.
> > > > It is very hard to keep track of which PRs and issues are waiting for
> > > user
> > > > feedback, have gone stale or are pending maintainer/committer action.
> > > > This means users frequently get no timely response, all the while we
> > > > regularly spend time on GH to look for PRs / issues needing action
> from
> > > us.
> > > > As a first step we should probably tackle PRs, once PRs are tackled
> and
> > > we
> > > > are satisfied with the solution, we can try to devise a similar one
> for
> > > GH
> > > > issues.
> > > >
> > > > An example of a great improvement is the CODEOWNERS addition [1].
> This
> > > > allows us to use filters like `is:pr is:open
> user-review-requested:@me
> > `
> > > [2]
> > > > which will show PRs that have requested a review from us. This does
> not
> > > > solve the problem of what are the PRs waiting for second review,
> > > > waiting for changes, etcetera.
> > > >
> > > > I don't think we have to reinvent the wheel, CPython has something
> that
> > > > works well and can easily be adapted/tweaked.
> > > > They use a GitHub bot (bedevere) with the following state machine:
> > > > https://github.com/python/bedevere#pr-state-machine
> > > >
> > > > PRs have one label of the following workflow labels, depending of the
> > > state:
> > > > - `Awaiting review`
> > > > - `Awaiting core review`
> > > > - `Awaiting changes`
> > > > - `Awaiting change review`
> > > > - `Awaiting merge`
> > > >
> > > > I would like to propose adding a GitHub bot to our repo that triggers
> > on
> > > PR
> > > > changes / comments implementing a similar workflow than the one on
> the
> > > > CPython repository.
> > > >
> > > > I am going to start working on it and I would love to hear feedback
> > about
> > > > that workflow. I have also created an issue on the Repo [3].
> > > >
> > > > Kind regards,
> > > > Raúl
> > > >
> > > > [1] https://github.com/apache/arrow/pull/33622
> > > > [2]
> > > >
> > >
> >
> https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+user-review-requested%3A%40me+
> > > > [3] https://github.com/apache/arrow/issues/33977
> > > >
> > >
> >
>

Re: [DISCUSS] PR automation workflow

[Nic Crane <th...@gmail.com>]

I have no specific comments on the what/how, other than to say I'm strongly
in favour of some kind of system being implemented and tried out, as I
currently rely on manual processes that are inefficient and make it easy to
miss PRs which need looking at.

On Thu, 2 Feb 2023 at 18:16, Andrew Lamb <al...@influxdata.com> wrote:

> A process that we use in arrow-rs / arrow-datafusion,  which is less
> precise but seems to be working well enough at the moment, is :
>
> 1. Mark  PRs that have received feedback and need more work prior to merge
> from `Ready to Review` back to `Draft`
> 2. Ask the author to set it back to "ready to review" when it is ready for
> the next round of review
>
> Andrew
>
>
> On Thu, Feb 2, 2023 at 4:17 AM Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Hi Raul,
> >
> > Since I'm the one who proposed that we reuse CPython's existing workflow
> > infrastructure, it follows logically that I'm in favour :-)
> >
> > I'm a CPython core developer myself (though inactive lately), I will add
> > that this workflow is really easing the work of reviewing PRs, as it
> > makes obvious whether a PR is needing attention from a committer.
> >
> > Once we start working with it, we may decide to make adjustments to
> > better fit our expectations, but I think we can start with the
> > unmodified workflow scheme.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 01/02/2023 à 15:34, Raúl Cumplido a écrit :
> > > Hi,
> > >
> > > I would like to start working on some automation for our PRs and issues
> > > workflows.
> > >
> > > I've heard, and have experienced, the frustration of spending a lot of
> > time
> > > on our issue tracker and our PRs to follow up on their status.
> > > It is very hard to keep track of which PRs and issues are waiting for
> > user
> > > feedback, have gone stale or are pending maintainer/committer action.
> > > This means users frequently get no timely response, all the while we
> > > regularly spend time on GH to look for PRs / issues needing action from
> > us.
> > > As a first step we should probably tackle PRs, once PRs are tackled and
> > we
> > > are satisfied with the solution, we can try to devise a similar one for
> > GH
> > > issues.
> > >
> > > An example of a great improvement is the CODEOWNERS addition [1]. This
> > > allows us to use filters like `is:pr is:open user-review-requested:@me
> `
> > [2]
> > > which will show PRs that have requested a review from us. This does not
> > > solve the problem of what are the PRs waiting for second review,
> > > waiting for changes, etcetera.
> > >
> > > I don't think we have to reinvent the wheel, CPython has something that
> > > works well and can easily be adapted/tweaked.
> > > They use a GitHub bot (bedevere) with the following state machine:
> > > https://github.com/python/bedevere#pr-state-machine
> > >
> > > PRs have one label of the following workflow labels, depending of the
> > state:
> > > - `Awaiting review`
> > > - `Awaiting core review`
> > > - `Awaiting changes`
> > > - `Awaiting change review`
> > > - `Awaiting merge`
> > >
> > > I would like to propose adding a GitHub bot to our repo that triggers
> on
> > PR
> > > changes / comments implementing a similar workflow than the one on the
> > > CPython repository.
> > >
> > > I am going to start working on it and I would love to hear feedback
> about
> > > that workflow. I have also created an issue on the Repo [3].
> > >
> > > Kind regards,
> > > Raúl
> > >
> > > [1] https://github.com/apache/arrow/pull/33622
> > > [2]
> > >
> >
> https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+user-review-requested%3A%40me+
> > > [3] https://github.com/apache/arrow/issues/33977
> > >
> >
>

Re: [DISCUSS] PR automation workflow

[Andrew Lamb <al...@influxdata.com>]

A process that we use in arrow-rs / arrow-datafusion,  which is less
precise but seems to be working well enough at the moment, is :

1. Mark  PRs that have received feedback and need more work prior to merge
from `Ready to Review` back to `Draft`
2. Ask the author to set it back to "ready to review" when it is ready for
the next round of review

Andrew


On Thu, Feb 2, 2023 at 4:17 AM Antoine Pitrou <an...@python.org> wrote:

>
> Hi Raul,
>
> Since I'm the one who proposed that we reuse CPython's existing workflow
> infrastructure, it follows logically that I'm in favour :-)
>
> I'm a CPython core developer myself (though inactive lately), I will add
> that this workflow is really easing the work of reviewing PRs, as it
> makes obvious whether a PR is needing attention from a committer.
>
> Once we start working with it, we may decide to make adjustments to
> better fit our expectations, but I think we can start with the
> unmodified workflow scheme.
>
> Regards
>
> Antoine.
>
>
> Le 01/02/2023 à 15:34, Raúl Cumplido a écrit :
> > Hi,
> >
> > I would like to start working on some automation for our PRs and issues
> > workflows.
> >
> > I've heard, and have experienced, the frustration of spending a lot of
> time
> > on our issue tracker and our PRs to follow up on their status.
> > It is very hard to keep track of which PRs and issues are waiting for
> user
> > feedback, have gone stale or are pending maintainer/committer action.
> > This means users frequently get no timely response, all the while we
> > regularly spend time on GH to look for PRs / issues needing action from
> us.
> > As a first step we should probably tackle PRs, once PRs are tackled and
> we
> > are satisfied with the solution, we can try to devise a similar one for
> GH
> > issues.
> >
> > An example of a great improvement is the CODEOWNERS addition [1]. This
> > allows us to use filters like `is:pr is:open user-review-requested:@me`
> [2]
> > which will show PRs that have requested a review from us. This does not
> > solve the problem of what are the PRs waiting for second review,
> > waiting for changes, etcetera.
> >
> > I don't think we have to reinvent the wheel, CPython has something that
> > works well and can easily be adapted/tweaked.
> > They use a GitHub bot (bedevere) with the following state machine:
> > https://github.com/python/bedevere#pr-state-machine
> >
> > PRs have one label of the following workflow labels, depending of the
> state:
> > - `Awaiting review`
> > - `Awaiting core review`
> > - `Awaiting changes`
> > - `Awaiting change review`
> > - `Awaiting merge`
> >
> > I would like to propose adding a GitHub bot to our repo that triggers on
> PR
> > changes / comments implementing a similar workflow than the one on the
> > CPython repository.
> >
> > I am going to start working on it and I would love to hear feedback about
> > that workflow. I have also created an issue on the Repo [3].
> >
> > Kind regards,
> > Raúl
> >
> > [1] https://github.com/apache/arrow/pull/33622
> > [2]
> >
> https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+user-review-requested%3A%40me+
> > [3] https://github.com/apache/arrow/issues/33977
> >
>

Re: [DISCUSS] PR automation workflow

[Antoine Pitrou <an...@python.org>]

Hi Raul,

Since I'm the one who proposed that we reuse CPython's existing workflow 
infrastructure, it follows logically that I'm in favour :-)

I'm a CPython core developer myself (though inactive lately), I will add 
that this workflow is really easing the work of reviewing PRs, as it 
makes obvious whether a PR is needing attention from a committer.

Once we start working with it, we may decide to make adjustments to 
better fit our expectations, but I think we can start with the 
unmodified workflow scheme.

Regards

Antoine.


Le 01/02/2023 à 15:34, Raúl Cumplido a écrit :
> Hi,
> 
> I would like to start working on some automation for our PRs and issues
> workflows.
> 
> I've heard, and have experienced, the frustration of spending a lot of time
> on our issue tracker and our PRs to follow up on their status.
> It is very hard to keep track of which PRs and issues are waiting for user
> feedback, have gone stale or are pending maintainer/committer action.
> This means users frequently get no timely response, all the while we
> regularly spend time on GH to look for PRs / issues needing action from us.
> As a first step we should probably tackle PRs, once PRs are tackled and we
> are satisfied with the solution, we can try to devise a similar one for GH
> issues.
> 
> An example of a great improvement is the CODEOWNERS addition [1]. This
> allows us to use filters like `is:pr is:open user-review-requested:@me` [2]
> which will show PRs that have requested a review from us. This does not
> solve the problem of what are the PRs waiting for second review,
> waiting for changes, etcetera.
> 
> I don't think we have to reinvent the wheel, CPython has something that
> works well and can easily be adapted/tweaked.
> They use a GitHub bot (bedevere) with the following state machine:
> https://github.com/python/bedevere#pr-state-machine
> 
> PRs have one label of the following workflow labels, depending of the state:
> - `Awaiting review`
> - `Awaiting core review`
> - `Awaiting changes`
> - `Awaiting change review`
> - `Awaiting merge`
> 
> I would like to propose adding a GitHub bot to our repo that triggers on PR
> changes / comments implementing a similar workflow than the one on the
> CPython repository.
> 
> I am going to start working on it and I would love to hear feedback about
> that workflow. I have also created an issue on the Repo [3].
> 
> Kind regards,
> Raúl
> 
> [1] https://github.com/apache/arrow/pull/33622
> [2]
> https://github.com/apache/arrow/pulls?q=is%3Apr+is%3Aopen+user-review-requested%3A%40me+
> [3] https://github.com/apache/arrow/issues/33977
>