Branching for Arrow releases

[Wes McKinney <we...@gmail.com>]

For the first few releases, we've been holding off merging patches to
master while the release vote is in progress, partially because of the
commits that the maven-release-plugin commits.

I would propose that in the future we continue to merge patches and perform
the release tag in a branch (so the release tag itself won't appear in the
master timeline) so that development flow is not interrupted. I'm not
familiar with what other projects having Java libraries do, so let me know
if there's a preferred workflow.

Thanks
Wes

Re: Branching for Arrow releases

[Holden Karau <ho...@pigscanfly.ca>]

I also think A is the best option :)

On Mon, May 8, 2017 at 11:42 AM, Bryan Cutler <cu...@gmail.com> wrote:

> I would prefer Option A, sounds like the easiest approach.  Only commits
> that are critical or have minimal risk would be cherry-picked to the
> release branch right?
>
> Bryan
>
> On Sat, May 6, 2017 at 3:18 PM, Wes McKinney <we...@gmail.com> wrote:
>
> > Out of the 3 options:
> >
> > A. Branching and cherry-picking commits until reaching an approved RC --
> at
> > point of branching, updating pom.xml to next version
> > B. Locking master during release votes
> > C. Merging PRs into a "dev-staging" branch, then rebasing on master after
> > release
> >
> > The one that is the simplest and interferes the least is A. B seems the
> > worst (given that we had ~300 patches in 10 weeks for the 0.3 release;
> this
> > is higher flow than normal but as the project grows that may become the
> new
> > normal). C would be OK but would require some tooling improvements in the
> > PR merge scripts (which is fine also).
> >
> > On Fri, May 5, 2017 at 3:03 PM, Julian Hyde <jh...@apache.org> wrote:
> >
> > > > There could be situations where we don't
> > > > want to include all commits in the next release candidate when a vote
> > > fails
> > >
> > > Absolutely agree. So, to adhere to Apache's policy of not rewriting
> > > master, you should make the release from a branch, but lock master so
> > > that you can fast-forward merge onto master when the release is
> > > approved.
> > >
> > > On Fri, May 5, 2017 at 11:48 AM, Wes McKinney <we...@gmail.com>
> > wrote:
> > > > I'm OK with rebasing master after releases for the next couple
> > releases,
> > > > and we'll see how it goes (though I had always thought force pushing
> > > > upstream master was a faux pas). There could be situations where we
> > don't
> > > > want to include all commits in the next release candidate when a vote
> > > fails:
> > > >
> > > > rcX
> > > > c1
> > > > c2
> > > > c3  <-- only need this commit
> > > > c4
> > > > c5
> > > > rc{X+ 1}
> > > >
> > > > This is purely hypothetical, so we can cross this bridge when we come
> > to
> > > > it.
> > > >
> > > > Thanks
> > > > Wes
> > > >
> > > > On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org>
> wrote:
> > > >
> > > >> I’m fine with either proposal (holding off commits during the
> release
> > > >> vote, or rebasing master afterwards).
> > > >>
> > > >> I agree with Julien that it’s really nice to have a simple, linear
> > > history
> > > >> (with releases on the master branch) and since Arrow is a fairly
> > > low-volume
> > > >> project we’re lucky we can do that.
> > > >>
> > > >> We’re in a similar situation in Calcite, and have arrived at a
> similar
> > > >> policy, where we tend to close master during the release quiet
> > period. I
> > > >> will often continue to process pull-requests; I stack them in a
> > personal
> > > >> branch called “next-master” and commit them all to master when
> master
> > > >> re-opens for commits.
> > > >>
> > > >> Julian
> > > >>
> > > >>
> > > >> > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com>
> > wrote:
> > > >> >
> > > >> > Alternatively we can rebase master on the release if patch have
> been
> > > >> merged
> > > >> > concurently to the release vote.
> > > >> > I think it is fine to rebase commits that have not been released
> > yet.
> > > >> > (the release sha however must stay the same)
> > > >> > I find usefull to have the release tag in the master history to
> know
> > > byt
> > > >> > looking at the git log if a given patch was before or after a
> > release.
> > > >> Even
> > > >> > if this info is duplicated somewhere else (jira) this one is the
> > > source
> > > >> of
> > > >> > truth.
> > > >> >
> > > >> > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <wesmckinn@gmail.com
> >
> > > >> wrote:
> > > >> >
> > > >> >> For the first few releases, we've been holding off merging
> patches
> > to
> > > >> >> master while the release vote is in progress, partially because
> of
> > > the
> > > >> >> commits that the maven-release-plugin commits.
> > > >> >>
> > > >> >> I would propose that in the future we continue to merge patches
> and
> > > >> perform
> > > >> >> the release tag in a branch (so the release tag itself won't
> appear
> > > in
> > > >> the
> > > >> >> master timeline) so that development flow is not interrupted. I'm
> > not
> > > >> >> familiar with what other projects having Java libraries do, so
> let
> > me
> > > >> know
> > > >> >> if there's a preferred workflow.
> > > >> >>
> > > >> >> Thanks
> > > >> >> Wes
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Julien
> > > >>
> > > >>
> > >
> >
>



-- 
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau

Re: Branching for Arrow releases

[Bryan Cutler <cu...@gmail.com>]

I would prefer Option A, sounds like the easiest approach.  Only commits
that are critical or have minimal risk would be cherry-picked to the
release branch right?

Bryan

On Sat, May 6, 2017 at 3:18 PM, Wes McKinney <we...@gmail.com> wrote:

> Out of the 3 options:
>
> A. Branching and cherry-picking commits until reaching an approved RC -- at
> point of branching, updating pom.xml to next version
> B. Locking master during release votes
> C. Merging PRs into a "dev-staging" branch, then rebasing on master after
> release
>
> The one that is the simplest and interferes the least is A. B seems the
> worst (given that we had ~300 patches in 10 weeks for the 0.3 release; this
> is higher flow than normal but as the project grows that may become the new
> normal). C would be OK but would require some tooling improvements in the
> PR merge scripts (which is fine also).
>
> On Fri, May 5, 2017 at 3:03 PM, Julian Hyde <jh...@apache.org> wrote:
>
> > > There could be situations where we don't
> > > want to include all commits in the next release candidate when a vote
> > fails
> >
> > Absolutely agree. So, to adhere to Apache's policy of not rewriting
> > master, you should make the release from a branch, but lock master so
> > that you can fast-forward merge onto master when the release is
> > approved.
> >
> > On Fri, May 5, 2017 at 11:48 AM, Wes McKinney <we...@gmail.com>
> wrote:
> > > I'm OK with rebasing master after releases for the next couple
> releases,
> > > and we'll see how it goes (though I had always thought force pushing
> > > upstream master was a faux pas). There could be situations where we
> don't
> > > want to include all commits in the next release candidate when a vote
> > fails:
> > >
> > > rcX
> > > c1
> > > c2
> > > c3  <-- only need this commit
> > > c4
> > > c5
> > > rc{X+ 1}
> > >
> > > This is purely hypothetical, so we can cross this bridge when we come
> to
> > > it.
> > >
> > > Thanks
> > > Wes
> > >
> > > On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org> wrote:
> > >
> > >> I’m fine with either proposal (holding off commits during the release
> > >> vote, or rebasing master afterwards).
> > >>
> > >> I agree with Julien that it’s really nice to have a simple, linear
> > history
> > >> (with releases on the master branch) and since Arrow is a fairly
> > low-volume
> > >> project we’re lucky we can do that.
> > >>
> > >> We’re in a similar situation in Calcite, and have arrived at a similar
> > >> policy, where we tend to close master during the release quiet
> period. I
> > >> will often continue to process pull-requests; I stack them in a
> personal
> > >> branch called “next-master” and commit them all to master when master
> > >> re-opens for commits.
> > >>
> > >> Julian
> > >>
> > >>
> > >> > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com>
> wrote:
> > >> >
> > >> > Alternatively we can rebase master on the release if patch have been
> > >> merged
> > >> > concurently to the release vote.
> > >> > I think it is fine to rebase commits that have not been released
> yet.
> > >> > (the release sha however must stay the same)
> > >> > I find usefull to have the release tag in the master history to know
> > byt
> > >> > looking at the git log if a given patch was before or after a
> release.
> > >> Even
> > >> > if this info is duplicated somewhere else (jira) this one is the
> > source
> > >> of
> > >> > truth.
> > >> >
> > >> > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com>
> > >> wrote:
> > >> >
> > >> >> For the first few releases, we've been holding off merging patches
> to
> > >> >> master while the release vote is in progress, partially because of
> > the
> > >> >> commits that the maven-release-plugin commits.
> > >> >>
> > >> >> I would propose that in the future we continue to merge patches and
> > >> perform
> > >> >> the release tag in a branch (so the release tag itself won't appear
> > in
> > >> the
> > >> >> master timeline) so that development flow is not interrupted. I'm
> not
> > >> >> familiar with what other projects having Java libraries do, so let
> me
> > >> know
> > >> >> if there's a preferred workflow.
> > >> >>
> > >> >> Thanks
> > >> >> Wes
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Julien
> > >>
> > >>
> >
>

Re: Branching for Arrow releases

[Wes McKinney <we...@gmail.com>]

Out of the 3 options:

A. Branching and cherry-picking commits until reaching an approved RC -- at
point of branching, updating pom.xml to next version
B. Locking master during release votes
C. Merging PRs into a "dev-staging" branch, then rebasing on master after
release

The one that is the simplest and interferes the least is A. B seems the
worst (given that we had ~300 patches in 10 weeks for the 0.3 release; this
is higher flow than normal but as the project grows that may become the new
normal). C would be OK but would require some tooling improvements in the
PR merge scripts (which is fine also).

On Fri, May 5, 2017 at 3:03 PM, Julian Hyde <jh...@apache.org> wrote:

> > There could be situations where we don't
> > want to include all commits in the next release candidate when a vote
> fails
>
> Absolutely agree. So, to adhere to Apache's policy of not rewriting
> master, you should make the release from a branch, but lock master so
> that you can fast-forward merge onto master when the release is
> approved.
>
> On Fri, May 5, 2017 at 11:48 AM, Wes McKinney <we...@gmail.com> wrote:
> > I'm OK with rebasing master after releases for the next couple releases,
> > and we'll see how it goes (though I had always thought force pushing
> > upstream master was a faux pas). There could be situations where we don't
> > want to include all commits in the next release candidate when a vote
> fails:
> >
> > rcX
> > c1
> > c2
> > c3  <-- only need this commit
> > c4
> > c5
> > rc{X+ 1}
> >
> > This is purely hypothetical, so we can cross this bridge when we come to
> > it.
> >
> > Thanks
> > Wes
> >
> > On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org> wrote:
> >
> >> I’m fine with either proposal (holding off commits during the release
> >> vote, or rebasing master afterwards).
> >>
> >> I agree with Julien that it’s really nice to have a simple, linear
> history
> >> (with releases on the master branch) and since Arrow is a fairly
> low-volume
> >> project we’re lucky we can do that.
> >>
> >> We’re in a similar situation in Calcite, and have arrived at a similar
> >> policy, where we tend to close master during the release quiet period. I
> >> will often continue to process pull-requests; I stack them in a personal
> >> branch called “next-master” and commit them all to master when master
> >> re-opens for commits.
> >>
> >> Julian
> >>
> >>
> >> > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com> wrote:
> >> >
> >> > Alternatively we can rebase master on the release if patch have been
> >> merged
> >> > concurently to the release vote.
> >> > I think it is fine to rebase commits that have not been released yet.
> >> > (the release sha however must stay the same)
> >> > I find usefull to have the release tag in the master history to know
> byt
> >> > looking at the git log if a given patch was before or after a release.
> >> Even
> >> > if this info is duplicated somewhere else (jira) this one is the
> source
> >> of
> >> > truth.
> >> >
> >> > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com>
> >> wrote:
> >> >
> >> >> For the first few releases, we've been holding off merging patches to
> >> >> master while the release vote is in progress, partially because of
> the
> >> >> commits that the maven-release-plugin commits.
> >> >>
> >> >> I would propose that in the future we continue to merge patches and
> >> perform
> >> >> the release tag in a branch (so the release tag itself won't appear
> in
> >> the
> >> >> master timeline) so that development flow is not interrupted. I'm not
> >> >> familiar with what other projects having Java libraries do, so let me
> >> know
> >> >> if there's a preferred workflow.
> >> >>
> >> >> Thanks
> >> >> Wes
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Julien
> >>
> >>
>

Re: Branching for Arrow releases

[Julian Hyde <jh...@apache.org>]

> There could be situations where we don't
> want to include all commits in the next release candidate when a vote fails

Absolutely agree. So, to adhere to Apache's policy of not rewriting
master, you should make the release from a branch, but lock master so
that you can fast-forward merge onto master when the release is
approved.

On Fri, May 5, 2017 at 11:48 AM, Wes McKinney <we...@gmail.com> wrote:
> I'm OK with rebasing master after releases for the next couple releases,
> and we'll see how it goes (though I had always thought force pushing
> upstream master was a faux pas). There could be situations where we don't
> want to include all commits in the next release candidate when a vote fails:
>
> rcX
> c1
> c2
> c3  <-- only need this commit
> c4
> c5
> rc{X+ 1}
>
> This is purely hypothetical, so we can cross this bridge when we come to
> it.
>
> Thanks
> Wes
>
> On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org> wrote:
>
>> I’m fine with either proposal (holding off commits during the release
>> vote, or rebasing master afterwards).
>>
>> I agree with Julien that it’s really nice to have a simple, linear history
>> (with releases on the master branch) and since Arrow is a fairly low-volume
>> project we’re lucky we can do that.
>>
>> We’re in a similar situation in Calcite, and have arrived at a similar
>> policy, where we tend to close master during the release quiet period. I
>> will often continue to process pull-requests; I stack them in a personal
>> branch called “next-master” and commit them all to master when master
>> re-opens for commits.
>>
>> Julian
>>
>>
>> > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com> wrote:
>> >
>> > Alternatively we can rebase master on the release if patch have been
>> merged
>> > concurently to the release vote.
>> > I think it is fine to rebase commits that have not been released yet.
>> > (the release sha however must stay the same)
>> > I find usefull to have the release tag in the master history to know byt
>> > looking at the git log if a given patch was before or after a release.
>> Even
>> > if this info is duplicated somewhere else (jira) this one is the source
>> of
>> > truth.
>> >
>> > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com>
>> wrote:
>> >
>> >> For the first few releases, we've been holding off merging patches to
>> >> master while the release vote is in progress, partially because of the
>> >> commits that the maven-release-plugin commits.
>> >>
>> >> I would propose that in the future we continue to merge patches and
>> perform
>> >> the release tag in a branch (so the release tag itself won't appear in
>> the
>> >> master timeline) so that development flow is not interrupted. I'm not
>> >> familiar with what other projects having Java libraries do, so let me
>> know
>> >> if there's a preferred workflow.
>> >>
>> >> Thanks
>> >> Wes
>> >>
>> >
>> >
>> >
>> > --
>> > Julien
>>
>>

Re: Branching for Arrow releases

[Jacques Nadeau <ja...@apache.org>]

I've never been a fan of the Calcite process where there is a strong desire
to keep the release commit in the master branch & locking the branch.

I prefer branching when we want to start a release. The branch can always
be cherry-picked to/from as necessary and/or re-forked as necessary. When
the branch occurs, the master get's bumped to the next snapshot version.

I prefer to avoid force-pushing on master.


On Fri, May 5, 2017 at 11:48 AM, Wes McKinney <we...@gmail.com> wrote:

> I'm OK with rebasing master after releases for the next couple releases,
> and we'll see how it goes (though I had always thought force pushing
> upstream master was a faux pas). There could be situations where we don't
> want to include all commits in the next release candidate when a vote
> fails:
>
> rcX
> c1
> c2
> c3  <-- only need this commit
> c4
> c5
> rc{X+ 1}
>
> This is purely hypothetical, so we can cross this bridge when we come to
> it.
>
> Thanks
> Wes
>
> On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org> wrote:
>
> > I’m fine with either proposal (holding off commits during the release
> > vote, or rebasing master afterwards).
> >
> > I agree with Julien that it’s really nice to have a simple, linear
> history
> > (with releases on the master branch) and since Arrow is a fairly
> low-volume
> > project we’re lucky we can do that.
> >
> > We’re in a similar situation in Calcite, and have arrived at a similar
> > policy, where we tend to close master during the release quiet period. I
> > will often continue to process pull-requests; I stack them in a personal
> > branch called “next-master” and commit them all to master when master
> > re-opens for commits.
> >
> > Julian
> >
> >
> > > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com> wrote:
> > >
> > > Alternatively we can rebase master on the release if patch have been
> > merged
> > > concurently to the release vote.
> > > I think it is fine to rebase commits that have not been released yet.
> > > (the release sha however must stay the same)
> > > I find usefull to have the release tag in the master history to know
> byt
> > > looking at the git log if a given patch was before or after a release.
> > Even
> > > if this info is duplicated somewhere else (jira) this one is the source
> > of
> > > truth.
> > >
> > > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com>
> > wrote:
> > >
> > >> For the first few releases, we've been holding off merging patches to
> > >> master while the release vote is in progress, partially because of the
> > >> commits that the maven-release-plugin commits.
> > >>
> > >> I would propose that in the future we continue to merge patches and
> > perform
> > >> the release tag in a branch (so the release tag itself won't appear in
> > the
> > >> master timeline) so that development flow is not interrupted. I'm not
> > >> familiar with what other projects having Java libraries do, so let me
> > know
> > >> if there's a preferred workflow.
> > >>
> > >> Thanks
> > >> Wes
> > >>
> > >
> > >
> > >
> > > --
> > > Julien
> >
> >
>

Re: Branching for Arrow releases

[Wes McKinney <we...@gmail.com>]

I'm OK with rebasing master after releases for the next couple releases,
and we'll see how it goes (though I had always thought force pushing
upstream master was a faux pas). There could be situations where we don't
want to include all commits in the next release candidate when a vote fails:

rcX
c1
c2
c3  <-- only need this commit
c4
c5
rc{X+ 1}

This is purely hypothetical, so we can cross this bridge when we come to
it.

Thanks
Wes

On Fri, May 5, 2017 at 12:51 PM, Julian Hyde <jh...@apache.org> wrote:

> I’m fine with either proposal (holding off commits during the release
> vote, or rebasing master afterwards).
>
> I agree with Julien that it’s really nice to have a simple, linear history
> (with releases on the master branch) and since Arrow is a fairly low-volume
> project we’re lucky we can do that.
>
> We’re in a similar situation in Calcite, and have arrived at a similar
> policy, where we tend to close master during the release quiet period. I
> will often continue to process pull-requests; I stack them in a personal
> branch called “next-master” and commit them all to master when master
> re-opens for commits.
>
> Julian
>
>
> > On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com> wrote:
> >
> > Alternatively we can rebase master on the release if patch have been
> merged
> > concurently to the release vote.
> > I think it is fine to rebase commits that have not been released yet.
> > (the release sha however must stay the same)
> > I find usefull to have the release tag in the master history to know byt
> > looking at the git log if a given patch was before or after a release.
> Even
> > if this info is duplicated somewhere else (jira) this one is the source
> of
> > truth.
> >
> > On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com>
> wrote:
> >
> >> For the first few releases, we've been holding off merging patches to
> >> master while the release vote is in progress, partially because of the
> >> commits that the maven-release-plugin commits.
> >>
> >> I would propose that in the future we continue to merge patches and
> perform
> >> the release tag in a branch (so the release tag itself won't appear in
> the
> >> master timeline) so that development flow is not interrupted. I'm not
> >> familiar with what other projects having Java libraries do, so let me
> know
> >> if there's a preferred workflow.
> >>
> >> Thanks
> >> Wes
> >>
> >
> >
> >
> > --
> > Julien
>
>

Re: Branching for Arrow releases

[Julian Hyde <jh...@apache.org>]

I’m fine with either proposal (holding off commits during the release vote, or rebasing master afterwards).

I agree with Julien that it’s really nice to have a simple, linear history (with releases on the master branch) and since Arrow is a fairly low-volume project we’re lucky we can do that.

We’re in a similar situation in Calcite, and have arrived at a similar policy, where we tend to close master during the release quiet period. I will often continue to process pull-requests; I stack them in a personal branch called “next-master” and commit them all to master when master re-opens for commits.

Julian
 

> On May 5, 2017, at 9:42 AM, Julien Le Dem <ju...@dremio.com> wrote:
> 
> Alternatively we can rebase master on the release if patch have been merged
> concurently to the release vote.
> I think it is fine to rebase commits that have not been released yet.
> (the release sha however must stay the same)
> I find usefull to have the release tag in the master history to know byt
> looking at the git log if a given patch was before or after a release. Even
> if this info is duplicated somewhere else (jira) this one is the source of
> truth.
> 
> On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com> wrote:
> 
>> For the first few releases, we've been holding off merging patches to
>> master while the release vote is in progress, partially because of the
>> commits that the maven-release-plugin commits.
>> 
>> I would propose that in the future we continue to merge patches and perform
>> the release tag in a branch (so the release tag itself won't appear in the
>> master timeline) so that development flow is not interrupted. I'm not
>> familiar with what other projects having Java libraries do, so let me know
>> if there's a preferred workflow.
>> 
>> Thanks
>> Wes
>> 
> 
> 
> 
> -- 
> Julien

Re: Branching for Arrow releases

[Julien Le Dem <ju...@dremio.com>]

Alternatively we can rebase master on the release if patch have been merged
concurently to the release vote.
I think it is fine to rebase commits that have not been released yet.
(the release sha however must stay the same)
I find usefull to have the release tag in the master history to know byt
looking at the git log if a given patch was before or after a release. Even
if this info is duplicated somewhere else (jira) this one is the source of
truth.

On Fri, May 5, 2017 at 7:18 AM, Wes McKinney <we...@gmail.com> wrote:

> For the first few releases, we've been holding off merging patches to
> master while the release vote is in progress, partially because of the
> commits that the maven-release-plugin commits.
>
> I would propose that in the future we continue to merge patches and perform
> the release tag in a branch (so the release tag itself won't appear in the
> master timeline) so that development flow is not interrupted. I'm not
> familiar with what other projects having Java libraries do, so let me know
> if there's a preferred workflow.
>
> Thanks
> Wes
>



-- 
Julien