You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by tison <wa...@gmail.com> on 2022/06/29 11:25:02 UTC
Queuing solution for merging patches
Hi,
There're several solutions around GitHub ecosystem to queuing patches
passed reviews and waiting for merged, especially in case to avoid semantic
conflict; e.g., Mergify or Bors-NG.
After enabled branch must be up-to-date with main branch, it introduces an
issue that multiple patches can race each other and cause unnecessary CI
tasks rerun - two patches can be verified simultaneously, after one merged,
the other should rerun, which generally cause O(n^2) task instance while
with queuing only O(n) is required.
Given that there're existing solutions, I'd like to ask what the best
practice for Apache projects host developments on GitHub on this topic. Or
what support does INFRA provide for the certain case.
Best,
tison.
Re: Queuing solution for merging patches
Posted by Jarek Potiuk <ja...@potiuk.com>.
Sorry wrong link: [2] - the discussion was moved to LEGAL and I asked it in
the legal one:
[2] https://issues.apache.org/jira/browse/LEGAL-599
On Wed, Jul 6, 2022 at 7:19 PM Jarek Potiuk <ja...@potiuk.com> wrote:
> I added my comment to [1]. I think there are good reasons why the decision
> about rejecting the "Merge Queue'' feature from GitHub should be revised
> (and I asked for it).
>
> IMHO it was partially based on misunderstanding how "merge queue" works
> and partially because we did not have the precedent. Now we have the
> Dependabot precedent which is IMHO basically doing the same thing, and I
> explained that explicit act of committer is still necessary for the merge
> queue to work (which was the basis for rejection).
>
> I keep my fingers crossed that we can have it as we also suffer from
> having to do back-forth with PRs
>
> [1] https://issues.apache.org/jira/browse/INFRA-22804
>
> On Thu, Jun 30, 2022 at 3:13 PM tison <wa...@gmail.com> wrote:
>
>> Hi Sheng,
>>
>> I think you're right that I can remove 'required_status_checks.strict' in
>> .asf.yaml to disable up-to-date requirements and then find other way to
>> resolve logic conflict issue. At least the committers should be aware of
>> which case should merge main branch.
>>
>> Hi Martinjn,
>>
>> Good to know that you meet similar requirements and those previous
>> discussions. I think one possible way is finding a sponsored bors-ng
>> deployment and ask INFRA to setup webhook, or ask INFRA to enable bors-ng /
>> mergify app on the repo. I don't know whether it's a good fit but some
>> thoughts.
>>
>> BTW, GitHub provides an option to always show "Update Branch" but not
>> require to be up-to-date. I'll ask INFRA whether there's an option to
>> configure. See also https://issues.apache.org/jira/browse/INFRA-23432.
>>
>> Best,
>> tison.
>>
>>
>> Martijn Visser <ma...@apache.org> 于2022年6月29日周三 21:13写道：
>>
>>> Hi Tison,
>>>
>>> I asked a couple of months ago [1] if Infra could enable Github's Merge
>>> Queue Functionality [2]. That was rejected unfortunately. I'm also
>>> curious
>>> if others think of a solution that would be compliant with the ASF rules.
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> [1] https://issues.apache.org/jira/browse/INFRA-22804
>>> [2]
>>>
>>> https://github.blog/changelog/2021-10-27-pull-request-merge-queue-limited-beta/
>>>
>>> Op wo 29 jun. 2022 om 13:53 schreef Sheng Wu <wu.sheng.841108@gmail.com
>>> >:
>>>
>>> > Hi
>>> >
>>> > I think the key is you set up to date for main branch, which makes CI
>>> has
>>> > to rerun.
>>> >
>>> > tison <wa...@gmail.com>于2022年6月29日 周三19:39写道：
>>> >
>>> >> Hi Sheng,
>>> >>
>>> >> Yes. I do _not_ ask INFRA to support it, but to see if there is
>>> existing
>>> >> practice.
>>> >>
>>> >> Best,
>>> >> tison.
>>> >>
>>> >>
>>> >> Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
>>> >>
>>> >> > Hi Tison
>>> >> >
>>> >> > I think there is no hard requirement from infra or Apache
>>> perspective.
>>> >> The
>>> >> > PMC could decide what they like, and ask Infra team to set them up.
>>> >> >
>>> >> > tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
>>> >> >
>>> >> > > Hi,
>>> >> > >
>>> >> > > There're several solutions around GitHub ecosystem to queuing
>>> patches
>>> >> > > passed reviews and waiting for merged, especially in case to avoid
>>> >> > semantic
>>> >> > > conflict; e.g., Mergify or Bors-NG.
>>> >> > >
>>> >> > > After enabled branch must be up-to-date with main branch, it
>>> >> introduces
>>> >> > an
>>> >> > > issue that multiple patches can race each other and cause
>>> unnecessary
>>> >> CI
>>> >> > > tasks rerun - two patches can be verified simultaneously, after
>>> one
>>> >> > merged,
>>> >> > > the other should rerun, which generally cause O(n^2) task instance
>>> >> while
>>> >> > > with queuing only O(n) is required.
>>> >> > >
>>> >> > > Given that there're existing solutions, I'd like to ask what the
>>> best
>>> >> > > practice for Apache projects host developments on GitHub on this
>>> >> topic.
>>> >> > Or
>>> >> > > what support does INFRA provide for the certain case.
>>> >> > >
>>> >> > > Best,
>>> >> > > tison.
>>> >> > >
>>> >> > --
>>> >> > Sheng Wu 吴晟
>>> >> >
>>> >> > Apache SkyWalking
>>> >> > Apache Incubator
>>> >> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
>>> >> > Zipkin
>>> >> > Twitter, wusheng1108
>>> >> >
>>> >>
>>> > --
>>> > Sheng Wu 吴晟
>>> >
>>> > Apache SkyWalking
>>> > Apache Incubator
>>> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
>>> > Zipkin
>>> > Twitter, wusheng1108
>>> >
>>>
>>
Re: Queuing solution for merging patches
Posted by Jarek Potiuk <ja...@potiuk.com>.
I added my comment to [1]. I think there are good reasons why the decision
about rejecting the "Merge Queue'' feature from GitHub should be revised
(and I asked for it).
IMHO it was partially based on misunderstanding how "merge queue" works and
partially because we did not have the precedent. Now we have the Dependabot
precedent which is IMHO basically doing the same thing, and I explained
that explicit act of committer is still necessary for the merge queue to
work (which was the basis for rejection).
I keep my fingers crossed that we can have it as we also suffer from having
to do back-forth with PRs
[1] https://issues.apache.org/jira/browse/INFRA-22804
On Thu, Jun 30, 2022 at 3:13 PM tison <wa...@gmail.com> wrote:
> Hi Sheng,
>
> I think you're right that I can remove 'required_status_checks.strict' in
> .asf.yaml to disable up-to-date requirements and then find other way to
> resolve logic conflict issue. At least the committers should be aware of
> which case should merge main branch.
>
> Hi Martinjn,
>
> Good to know that you meet similar requirements and those previous
> discussions. I think one possible way is finding a sponsored bors-ng
> deployment and ask INFRA to setup webhook, or ask INFRA to enable bors-ng /
> mergify app on the repo. I don't know whether it's a good fit but some
> thoughts.
>
> BTW, GitHub provides an option to always show "Update Branch" but not
> require to be up-to-date. I'll ask INFRA whether there's an option to
> configure. See also https://issues.apache.org/jira/browse/INFRA-23432.
>
> Best,
> tison.
>
>
> Martijn Visser <ma...@apache.org> 于2022年6月29日周三 21:13写道：
>
>> Hi Tison,
>>
>> I asked a couple of months ago [1] if Infra could enable Github's Merge
>> Queue Functionality [2]. That was rejected unfortunately. I'm also curious
>> if others think of a solution that would be compliant with the ASF rules.
>>
>> Best regards,
>>
>> Martijn
>>
>> [1] https://issues.apache.org/jira/browse/INFRA-22804
>> [2]
>>
>> https://github.blog/changelog/2021-10-27-pull-request-merge-queue-limited-beta/
>>
>> Op wo 29 jun. 2022 om 13:53 schreef Sheng Wu <wu...@gmail.com>:
>>
>> > Hi
>> >
>> > I think the key is you set up to date for main branch, which makes CI
>> has
>> > to rerun.
>> >
>> > tison <wa...@gmail.com>于2022年6月29日 周三19:39写道：
>> >
>> >> Hi Sheng,
>> >>
>> >> Yes. I do _not_ ask INFRA to support it, but to see if there is
>> existing
>> >> practice.
>> >>
>> >> Best,
>> >> tison.
>> >>
>> >>
>> >> Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
>> >>
>> >> > Hi Tison
>> >> >
>> >> > I think there is no hard requirement from infra or Apache
>> perspective.
>> >> The
>> >> > PMC could decide what they like, and ask Infra team to set them up.
>> >> >
>> >> > tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
>> >> >
>> >> > > Hi,
>> >> > >
>> >> > > There're several solutions around GitHub ecosystem to queuing
>> patches
>> >> > > passed reviews and waiting for merged, especially in case to avoid
>> >> > semantic
>> >> > > conflict; e.g., Mergify or Bors-NG.
>> >> > >
>> >> > > After enabled branch must be up-to-date with main branch, it
>> >> introduces
>> >> > an
>> >> > > issue that multiple patches can race each other and cause
>> unnecessary
>> >> CI
>> >> > > tasks rerun - two patches can be verified simultaneously, after one
>> >> > merged,
>> >> > > the other should rerun, which generally cause O(n^2) task instance
>> >> while
>> >> > > with queuing only O(n) is required.
>> >> > >
>> >> > > Given that there're existing solutions, I'd like to ask what the
>> best
>> >> > > practice for Apache projects host developments on GitHub on this
>> >> topic.
>> >> > Or
>> >> > > what support does INFRA provide for the certain case.
>> >> > >
>> >> > > Best,
>> >> > > tison.
>> >> > >
>> >> > --
>> >> > Sheng Wu 吴晟
>> >> >
>> >> > Apache SkyWalking
>> >> > Apache Incubator
>> >> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
>> >> > Zipkin
>> >> > Twitter, wusheng1108
>> >> >
>> >>
>> > --
>> > Sheng Wu 吴晟
>> >
>> > Apache SkyWalking
>> > Apache Incubator
>> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
>> > Zipkin
>> > Twitter, wusheng1108
>> >
>>
>
Re: Queuing solution for merging patches
Posted by tison <wa...@gmail.com>.
Hi Sheng,
I think you're right that I can remove 'required_status_checks.strict' in
.asf.yaml to disable up-to-date requirements and then find other way to
resolve logic conflict issue. At least the committers should be aware of
which case should merge main branch.
Hi Martinjn,
Good to know that you meet similar requirements and those previous
discussions. I think one possible way is finding a sponsored bors-ng
deployment and ask INFRA to setup webhook, or ask INFRA to enable bors-ng /
mergify app on the repo. I don't know whether it's a good fit but some
thoughts.
BTW, GitHub provides an option to always show "Update Branch" but not
require to be up-to-date. I'll ask INFRA whether there's an option to
configure. See also https://issues.apache.org/jira/browse/INFRA-23432.
Best,
tison.
Martijn Visser <ma...@apache.org> 于2022年6月29日周三 21:13写道：
> Hi Tison,
>
> I asked a couple of months ago [1] if Infra could enable Github's Merge
> Queue Functionality [2]. That was rejected unfortunately. I'm also curious
> if others think of a solution that would be compliant with the ASF rules.
>
> Best regards,
>
> Martijn
>
> [1] https://issues.apache.org/jira/browse/INFRA-22804
> [2]
>
> https://github.blog/changelog/2021-10-27-pull-request-merge-queue-limited-beta/
>
> Op wo 29 jun. 2022 om 13:53 schreef Sheng Wu <wu...@gmail.com>:
>
> > Hi
> >
> > I think the key is you set up to date for main branch, which makes CI has
> > to rerun.
> >
> > tison <wa...@gmail.com>于2022年6月29日 周三19:39写道：
> >
> >> Hi Sheng,
> >>
> >> Yes. I do _not_ ask INFRA to support it, but to see if there is existing
> >> practice.
> >>
> >> Best,
> >> tison.
> >>
> >>
> >> Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
> >>
> >> > Hi Tison
> >> >
> >> > I think there is no hard requirement from infra or Apache perspective.
> >> The
> >> > PMC could decide what they like, and ask Infra team to set them up.
> >> >
> >> > tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
> >> >
> >> > > Hi,
> >> > >
> >> > > There're several solutions around GitHub ecosystem to queuing
> patches
> >> > > passed reviews and waiting for merged, especially in case to avoid
> >> > semantic
> >> > > conflict; e.g., Mergify or Bors-NG.
> >> > >
> >> > > After enabled branch must be up-to-date with main branch, it
> >> introduces
> >> > an
> >> > > issue that multiple patches can race each other and cause
> unnecessary
> >> CI
> >> > > tasks rerun - two patches can be verified simultaneously, after one
> >> > merged,
> >> > > the other should rerun, which generally cause O(n^2) task instance
> >> while
> >> > > with queuing only O(n) is required.
> >> > >
> >> > > Given that there're existing solutions, I'd like to ask what the
> best
> >> > > practice for Apache projects host developments on GitHub on this
> >> topic.
> >> > Or
> >> > > what support does INFRA provide for the certain case.
> >> > >
> >> > > Best,
> >> > > tison.
> >> > >
> >> > --
> >> > Sheng Wu 吴晟
> >> >
> >> > Apache SkyWalking
> >> > Apache Incubator
> >> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
> >> > Zipkin
> >> > Twitter, wusheng1108
> >> >
> >>
> > --
> > Sheng Wu 吴晟
> >
> > Apache SkyWalking
> > Apache Incubator
> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
> > Zipkin
> > Twitter, wusheng1108
> >
>
Re: Queuing solution for merging patches
Posted by Martijn Visser <ma...@apache.org>.
Hi Tison,
I asked a couple of months ago [1] if Infra could enable Github's Merge
Queue Functionality [2]. That was rejected unfortunately. I'm also curious
if others think of a solution that would be compliant with the ASF rules.
Best regards,
Martijn
[1] https://issues.apache.org/jira/browse/INFRA-22804
[2]
https://github.blog/changelog/2021-10-27-pull-request-merge-queue-limited-beta/
Op wo 29 jun. 2022 om 13:53 schreef Sheng Wu <wu...@gmail.com>:
> Hi
>
> I think the key is you set up to date for main branch, which makes CI has
> to rerun.
>
> tison <wa...@gmail.com>于2022年6月29日 周三19:39写道：
>
>> Hi Sheng,
>>
>> Yes. I do _not_ ask INFRA to support it, but to see if there is existing
>> practice.
>>
>> Best,
>> tison.
>>
>>
>> Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
>>
>> > Hi Tison
>> >
>> > I think there is no hard requirement from infra or Apache perspective.
>> The
>> > PMC could decide what they like, and ask Infra team to set them up.
>> >
>> > tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
>> >
>> > > Hi,
>> > >
>> > > There're several solutions around GitHub ecosystem to queuing patches
>> > > passed reviews and waiting for merged, especially in case to avoid
>> > semantic
>> > > conflict; e.g., Mergify or Bors-NG.
>> > >
>> > > After enabled branch must be up-to-date with main branch, it
>> introduces
>> > an
>> > > issue that multiple patches can race each other and cause unnecessary
>> CI
>> > > tasks rerun - two patches can be verified simultaneously, after one
>> > merged,
>> > > the other should rerun, which generally cause O(n^2) task instance
>> while
>> > > with queuing only O(n) is required.
>> > >
>> > > Given that there're existing solutions, I'd like to ask what the best
>> > > practice for Apache projects host developments on GitHub on this
>> topic.
>> > Or
>> > > what support does INFRA provide for the certain case.
>> > >
>> > > Best,
>> > > tison.
>> > >
>> > --
>> > Sheng Wu 吴晟
>> >
>> > Apache SkyWalking
>> > Apache Incubator
>> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
>> > Zipkin
>> > Twitter, wusheng1108
>> >
>>
> --
> Sheng Wu 吴晟
>
> Apache SkyWalking
> Apache Incubator
> Apache ShardingSphere, ECharts, DolphinScheduler podlings
> Zipkin
> Twitter, wusheng1108
>
Re: Queuing solution for merging patches
Posted by Sheng Wu <wu...@gmail.com>.
Hi
I think the key is you set up to date for main branch, which makes CI has
to rerun.
tison <wa...@gmail.com>于2022年6月29日 周三19:39写道：
> Hi Sheng,
>
> Yes. I do _not_ ask INFRA to support it, but to see if there is existing
> practice.
>
> Best,
> tison.
>
>
> Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
>
> > Hi Tison
> >
> > I think there is no hard requirement from infra or Apache perspective.
> The
> > PMC could decide what they like, and ask Infra team to set them up.
> >
> > tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
> >
> > > Hi,
> > >
> > > There're several solutions around GitHub ecosystem to queuing patches
> > > passed reviews and waiting for merged, especially in case to avoid
> > semantic
> > > conflict; e.g., Mergify or Bors-NG.
> > >
> > > After enabled branch must be up-to-date with main branch, it introduces
> > an
> > > issue that multiple patches can race each other and cause unnecessary
> CI
> > > tasks rerun - two patches can be verified simultaneously, after one
> > merged,
> > > the other should rerun, which generally cause O(n^2) task instance
> while
> > > with queuing only O(n) is required.
> > >
> > > Given that there're existing solutions, I'd like to ask what the best
> > > practice for Apache projects host developments on GitHub on this topic.
> > Or
> > > what support does INFRA provide for the certain case.
> > >
> > > Best,
> > > tison.
> > >
> > --
> > Sheng Wu 吴晟
> >
> > Apache SkyWalking
> > Apache Incubator
> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
> > Zipkin
> > Twitter, wusheng1108
> >
>
--
Sheng Wu 吴晟
Apache SkyWalking
Apache Incubator
Apache ShardingSphere, ECharts, DolphinScheduler podlings
Zipkin
Twitter, wusheng1108
Re: Queuing solution for merging patches
Posted by tison <wa...@gmail.com>.
Hi Sheng,
Yes. I do _not_ ask INFRA to support it, but to see if there is existing
practice.
Best,
tison.
Sheng Wu <wu...@gmail.com> 于2022年6月29日周三 19:33写道：
> Hi Tison
>
> I think there is no hard requirement from infra or Apache perspective. The
> PMC could decide what they like, and ask Infra team to set them up.
>
> tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
>
> > Hi,
> >
> > There're several solutions around GitHub ecosystem to queuing patches
> > passed reviews and waiting for merged, especially in case to avoid
> semantic
> > conflict; e.g., Mergify or Bors-NG.
> >
> > After enabled branch must be up-to-date with main branch, it introduces
> an
> > issue that multiple patches can race each other and cause unnecessary CI
> > tasks rerun - two patches can be verified simultaneously, after one
> merged,
> > the other should rerun, which generally cause O(n^2) task instance while
> > with queuing only O(n) is required.
> >
> > Given that there're existing solutions, I'd like to ask what the best
> > practice for Apache projects host developments on GitHub on this topic.
> Or
> > what support does INFRA provide for the certain case.
> >
> > Best,
> > tison.
> >
> --
> Sheng Wu 吴晟
>
> Apache SkyWalking
> Apache Incubator
> Apache ShardingSphere, ECharts, DolphinScheduler podlings
> Zipkin
> Twitter, wusheng1108
>
Re: Queuing solution for merging patches
Posted by Sheng Wu <wu...@gmail.com>.
Hi Tison
I think there is no hard requirement from infra or Apache perspective. The
PMC could decide what they like, and ask Infra team to set them up.
tison <wa...@gmail.com>于2022年6月29日 周三19:25写道：
> Hi,
>
> There're several solutions around GitHub ecosystem to queuing patches
> passed reviews and waiting for merged, especially in case to avoid semantic
> conflict; e.g., Mergify or Bors-NG.
>
> After enabled branch must be up-to-date with main branch, it introduces an
> issue that multiple patches can race each other and cause unnecessary CI
> tasks rerun - two patches can be verified simultaneously, after one merged,
> the other should rerun, which generally cause O(n^2) task instance while
> with queuing only O(n) is required.
>
> Given that there're existing solutions, I'd like to ask what the best
> practice for Apache projects host developments on GitHub on this topic. Or
> what support does INFRA provide for the certain case.
>
> Best,
> tison.
>
--
Sheng Wu 吴晟
Apache SkyWalking
Apache Incubator
Apache ShardingSphere, ECharts, DolphinScheduler podlings
Zipkin
Twitter, wusheng1108