You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@geode.apache.org by Alexander Murmann <am...@vmware.com> on 2022/01/26 01:08:53 UTC

Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Hi everyone,

Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Robert Houghton <rh...@vmware.com>.

A third option, is to cut the branch as normal, and be ready to revert if we see some kind of issue as we are stabilizing the redis changes

On Jan 25, 2022 17:10, Alexander Murmann <am...@vmware.com> wrote:
Hi everyone,

Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Darrel Schneider <da...@vmware.com>.

I think we will probably always have this struggle around the time we cut a new release branch and then need to decide what changes on develop to also bring to the release branch. I'm okay with us not including the big cleanup in 1.15 since it is not something we plan on backporting to other support branches and if we had cut the 1.15 release branch right before this change, I don't think we would have approved it to be back ported to the 1.15 release branch.

I share Owen's concern that not including it might case extra work in resolving conflicts in every change to develop after it that we also want to include in 1.15. We will not know if this is true until we try cherry-picking those revisions. So I'm unsure what the best way of not including it is. I'm okay with us cutting the branch from the revision before the big commit and then working through the list of commits after it and deciding which need to be in the 1.15 release.
________________________________
From: Alexander Murmann <am...@vmware.com>
Sent: Wednesday, January 26, 2022 7:21 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Owen, I really appreciate your point about the increased cost of backports by the branches diverging like this. I do wonder how high the cost will be in practice, given that AFAIK most of these changes limit themselves to a single line.
________________________________
From: Owen Nichols <on...@vmware.com>
Sent: Tuesday, January 25, 2022 20:18
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

    Hi everyone,

    Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

    Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Owen Nichols <on...@vmware.com>.

Thanks Ray for proposing this branch and Alexander for suggesting a branchpoint. I will volunteer as Release Manager.

** The support/1.15 branch has now been cut from the SHA proposed in this thread and develop is now 1.16. **

Please use your best judgement in determining what to backport.

PRs are welcome, but optional! Committers, please merge your own PRs.

If your backport will take some while, you can add the "blocks-1.15.0" label in Jira to let others know.

Please do raise any concerns on the dev list and/or via the blocks-1.15.0 label.

Please do test the branch as much as possible to make this the best release of Geode.

-Owen

On 1/26/22, 10:16 AM, "Raymond Ingles" <ri...@vmware.com> wrote:

BTW, just to clarify, when I officially proposed cutting the branch, I hadn't intended to volunteer as release manager this round. That said, it's important to branch at a point we're confident about.

Bisecting a 3K-file change is potentially... complicated. If there's confidence we can track down the issue(s) quickly, a later branch point would be nice. If not... probably better to branch at a "not known bad" point.

On 1/26/22, 1:03 PM, "Mark Hanson" <ha...@vmware.com> wrote:

First, I think I would suggest that we have someone cut a branch as suggested and see how long it actually takes.

Second, I would suggest we define a norm if we want to avoid this in the future.

Third, I don't really like the risk of having this in, but I have only heard about performance changes in our performance testing. Is there a specific defect? I looked at the code changes (not all 3000 files but a chunk) before I approved them. The changes were mostly dealing with warnings like unboxing etc. Given that these types of changes are lower risk individually, though obviously of concern en masse, I would like to see a bug or something before we decide to increase the work load by branching before the change. I understand that we are nervous about creating a release, but let's get some data (bugs) to justify the effort.

Thanks,
Mark

On 1/26/22, 7:21 AM, "Alexander Murmann" <am...@vmware.com> wrote:

Owen, I really appreciate your point about the increased cost of backports by the branches diverging like this. I do wonder how high the cost will be in practice, given that AFAIK most of these changes limit themselves to a single line.
________________________________
From: Owen Nichols <on...@vmware.com>
Sent: Tuesday, January 25, 2022 20:18
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

Hi everyone,

Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Mark Hanson <ha...@vmware.com>.

To be clear, I agree with Donal and Robert.

On 1/26/22, 10:44 AM, "Donal Evans" <do...@vmware.com> wrote:

To clarify what I think might be a misunderstanding here, there is zero evidence of any kind that the large warning clean-up PR has introduced any issues, performance-related or otherwise.

For my two cents on cutting the branch at the commit before the big change, I'm of the same opinion as Robert, that we should cut the branch at HEAD and revert only if it's determined to be necessary. The changes were broad, but the nature of the changes were benign and low-risk, being simple, automated refactors of trivial things like removing unnecessary casts or adding "final" to variables.

I understand the argument that we often don't see all the consequences of a change until some time after it's been merged, but that argument applies to every change that gets made, and historically, new features and bug fixes have a far higher change of introducing problems than automated refactors. The choice of the warning clean-up change seems based almost entirely on the fact that it touched a lot of files, not in the actual risk associated with those changes.
________________________________
From: Raymond Ingles <ri...@vmware.com>
Sent: Wednesday, January 26, 2022 10:16 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

On 1/26/22, 1:03 PM, "Mark Hanson" <ha...@vmware.com> wrote:

First, I think I would suggest that we have someone cut a branch as suggested and see how long it actually takes.

Second, I would suggest we define a norm if we want to avoid this in the future.

Thanks,
Mark

On 1/26/22, 7:21 AM, "Alexander Murmann" <am...@vmware.com> wrote:

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

Hi everyone,

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Donal Evans <do...@vmware.com>.

To clarify what I think might be a misunderstanding here, there is zero evidence of any kind that the large warning clean-up PR has introduced any issues, performance-related or otherwise.

For my two cents on cutting the branch at the commit before the big change, I'm of the same opinion as Robert, that we should cut the branch at HEAD and revert only if it's determined to be necessary. The changes were broad, but the nature of the changes were benign and low-risk, being simple, automated refactors of trivial things like removing unnecessary casts or adding "final" to variables.

I understand the argument that we often don't see all the consequences of a change until some time after it's been merged, but that argument applies to every change that gets made, and historically, new features and bug fixes have a far higher change of introducing problems than automated refactors. The choice of the warning clean-up change seems based almost entirely on the fact that it touched a lot of files, not in the actual risk associated with those changes.
________________________________
From: Raymond Ingles <ri...@vmware.com>
Sent: Wednesday, January 26, 2022 10:16 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

BTW, just to clarify, when I officially proposed cutting the branch, I hadn't intended to volunteer as release manager this round. That said, it's important to branch at a point we're confident about.

Bisecting a 3K-file change is potentially... complicated. If there's confidence we can track down the issue(s) quickly, a later branch point would be nice. If not... probably better to branch at a "not known bad" point.

On 1/26/22, 1:03 PM, "Mark Hanson" <ha...@vmware.com> wrote:

    First, I think I would suggest that we have someone cut a branch as suggested and see how long it actually takes.

    Second, I would suggest we define a norm if we want to avoid this in the future.

    Third, I don't really like the risk of having this in, but I have only heard about performance changes in our performance testing. Is there a specific defect? I looked at the code changes (not all 3000 files but a chunk) before I approved them. The changes were mostly dealing with warnings like unboxing etc. Given that these types of changes are lower risk individually, though obviously of concern en masse, I would like to see a bug or something before we decide to increase the work load by branching before the change. I understand that we are nervous about creating a release, but let's get some data (bugs) to justify the effort.

    Thanks,
    Mark

    On 1/26/22, 7:21 AM, "Alexander Murmann" <am...@vmware.com> wrote:

        Owen, I really appreciate your point about the increased cost of backports by the branches diverging like this. I do wonder how high the cost will be in practice, given that AFAIK most of these changes limit themselves to a single line.
        ________________________________
        From: Owen Nichols <on...@vmware.com>
        Sent: Tuesday, January 25, 2022 20:18
        To: dev@geode.apache.org <de...@geode.apache.org>
        Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

        Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

        Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

        On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

            Hi everyone,

            Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

            Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Raymond Ingles <ri...@vmware.com>.

On 1/26/22, 1:03 PM, "Mark Hanson" <ha...@vmware.com> wrote:

First, I think I would suggest that we have someone cut a branch as suggested and see how long it actually takes.

Second, I would suggest we define a norm if we want to avoid this in the future.

Thanks,
Mark

On 1/26/22, 7:21 AM, "Alexander Murmann" <am...@vmware.com> wrote:

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

Hi everyone,

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Mark Hanson <ha...@vmware.com>.

First, I think I would suggest that we have someone cut a branch as suggested and see how long it actually takes.

Second, I would suggest we define a norm if we want to avoid this in the future.

Thanks,
Mark

On 1/26/22, 7:21 AM, "Alexander Murmann" <am...@vmware.com> wrote:

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

Hi everyone,

Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Alexander Murmann <am...@vmware.com>.

Owen, I really appreciate your point about the increased cost of backports by the branches diverging like this. I do wonder how high the cost will be in practice, given that AFAIK most of these changes limit themselves to a single line.
________________________________
From: Owen Nichols <on...@vmware.com>
Sent: Tuesday, January 25, 2022 20:18
To: dev@geode.apache.org <de...@geode.apache.org>
Subject: Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

    Hi everyone,

    Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

    Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Anthony Baker <ba...@vmware.com>.

Fair point, although my hope is that we continually and incrementally improve our code base on the /develop branch. Yes, there is an increased cost of back porting critical changes to support branches but the tradeoff is worth it IMHO.

Anthony

> On Jan 25, 2022, at 11:18 PM, Owen Nichols <on...@vmware.com> wrote:
> 
> Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored".

Re: Proposal: Cut 1.15 release branch from SHA 8f7193c827ee3198ae374101221c02039c70a561

Posted by Owen Nichols <on...@vmware.com>.

Even a small change can have subtle but important effects only discovered after a long time, so leaning on commit-size as a proxy for risk may only serve to create a false sense of security.

Also to consider, having a large refactor on develop but not support/1.15 will increase backporting pain, as many cherry-picks will have merge conflicts that have to be manually "un-refactored". 

On 1/25/22, 5:09 PM, "Alexander Murmann" <am...@vmware.com> wrote:

    Hi everyone,

    Last week we discussed to cut the 1.15 release branch. I would like to propose that we cut the branch from last week's SHA 8f7193c827ee3198ae374101221c02039c70a561. The following commit is a very large refactor. Nothing obvious seems wrong with that change, but given that we frequently only discover very subtle, but important changes to Geode after a long time, I think that this would allow us to reduce some risk for 1.15 and its future users and give this large change some time to proof itself on the develop branch. I'd love to avoid that work entirely, but am concerned that we only might find out about problems a few weeks from now or worse, after we shipped.

    Another option might be to branch from head and revert the change on the release branch. I am uncertain which approach will proof less work.