Re: [Discussion] Clarify the support story for released Beam versions

[Ahmet Altay <al...@google.com>]

Sorry for the late reply, I was out of office for a while.

On Thu, Aug 23, 2018 at 3:21 PM, Andrew Pilloud <ap...@google.com> wrote:

> It would be good to gather data on who the users will be and what they
> expect out of it. Beam users appear to be tech savvy early adopters, and
> Enterprise users of that class frequently maintain their own patch stacks
> of backported fixes and features off of releases. I've been in that world
> before as a user (of other open source projects but not of Beam). It
> wouldn't surprise me if several of those patch stacks are on github forks
> of Beam. Hopefully we can track some of these down as an example of where
> we can reduce pain for our users.
>

Any suggestions on how we can gather this information? We tried to reach
out to users earlier in the process but did not hear much back from them.


>
> Andrew
>
>
> On Thu, Aug 23, 2018, 1:56 AM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Hi,
>> I agree that LTS releases are a good thing for users especially because
>> of the argument given by Ahmet (enterprise users). Just 2 comments:
>> - It will require a good amount of backports
>>
> I agree this is a risk, however I am hoping that we can limit to major
issues and there will be a limited number of those.


> - The LTS frequency needs to be flexible IMHO but we must make sure the
>> period between two LTS is acceptable.
>>
> I believe our most recent iteration (i.e. community decides case by case
basis and there will be one at least every 12 months) satisfies this
requirement.



>
>> Etienne
>>
>> Le jeudi 16 août 2018 à 23:13 +0200, Alexey Romanenko a écrit :
>>
>> On 16 Aug 2018, at 21:10, Robert Bradshaw <ro...@google.com> wrote:
>>
>> On Thu, Aug 16, 2018 at 7:56 PM Ahmet Altay <al...@google.com> wrote:
>>
>> I agree with this assessment and the risk. I proposed every Nth as a way
>> to keep LTS releases coming out. I was concerned about not making any LTS
>> releases for a long time.
>>
>>
>> Yeah. We could certainly have every Nth release by default to keep the
>> cadence up, but note that it's very flexible. Your wording below
>> sounds fine to me.
>>
>>
>> This sounds reasonable for me too, thanks.
>>
>>
>> How about something like:
>>
>> "The community will mark some releases LTS releases (based on the factors
>> such as the number of LTS releases currently in flight, and whether the
>> accumulated feature set from the last LTS provides significant value to
>> upgrade). There will be at least one new LTS release in a 12 month period."
>>
>> I prepared a PR draft for the above change (https://github.com/apache/
>> beam-site/pull/539). I will be OOO for a few weeks, feel free to
>> edit/merge my PR. I can also do this when I am back.
>>
>>
>>
>> Generally, x.0.0 releases are not used by the target audiences of LTS
>> releases, so I would not plan on it (by default at least) becoming an LTS
>> candidate.
>>
>>
>>
>> I agree with Robert. We can let the community decide based on how we feel
>> at that time, but unlikely in my opinion.
>>
>>
>>
>>
>> On Thursday, August 16, 2018, Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>>
>> Ahmet, thank you for raising this topic, I think it defenitevly makes
>> sense to have LTS releases, especially for enterprise users. The other
>> potential solution could be patching only last 2-3 releases but, with a
>> goal of 8 releases per year, it might cover quite a short time slice.
>>
>> The only question for me - do we consider major release (3.0 will be the
>> next one) as LTS release by default despite of the its number in release
>> sequence?
>>
>> On 15 Aug 2018, at 20:36, Pablo Estrada <pa...@google.com> wrote:
>>
>> No, I think that sounds good : )
>>
>> On Wed, Aug 15, 2018 at 11:34 AM Ahmet Altay <al...@google.com> wrote:
>>
>>
>> In the PR, I proposed starting with the next (2.7.0) release. I should
>> have made it more clear. One way of tracking would be having a table, or
>> perhaps adding this information to the downloads page. Do you have any
>> ideas?
>>
>>
>> On Wed, Aug 15, 2018 at 11:28 AM, Pablo Estrada <pa...@google.com>
>> wrote:
>>
>>
>> Thanks Ahmet for looking into this. I have a follow-up question. Have you
>> thought about the next few releases, and which one will be the first LTS
>> release? Also, how should we track this?
>> -P.
>>
>> On Wed, Aug 15, 2018 at 11:24 AM Ahmet Altay <al...@google.com> wrote:
>>
>>
>> PR is reviewed and merged. Thank you all for the input. If you did not
>> have a chance to share your feedback, please propose any modifications that
>> you would like to see. I will be more than happy to make changes that would
>> allows us to serve our users better.
>>
>> On Tue, Aug 14, 2018 at 4:32 PM, Ahmet Altay <al...@google.com> wrote:
>>
>>
>> Still waiting for any additional user feedback to come. I added reviewers
>> to the PR. Unless there is objections or additional feedback I would like
>> to go ahead with this version as it is. Modifications after that would
>> always be welcome.
>>
>> On Mon, Aug 13, 2018 at 2:06 PM, Rafael Fernandez <rf...@google.com>
>> wrote:
>>
>>
>> I think this will great for the project! It's worked well for others
>> (such as Ubuntu). I like that this remains compatible with our desire to
>> release every six weeks, while keeping the support/patch load manageable.
>>
>> Release: +1 single process. This is just a statement of what we commit to
>> service.
>>
>> On Mon, Aug 13, 2018 at 12:31 PM Ahmet Altay <al...@google.com> wrote:
>>
>>
>> I was not proposing any additional changes to the release process. If we
>> think that release process could be improved it would make sense to apply
>> it to all releases.
>>
>> On Mon, Aug 13, 2018 at 11:01 AM, Lukasz Cwik <lc...@google.com> wrote:
>>
>>
>> Charles, I would keep the process the same with respect to releasing.
>>
>> On Mon, Aug 13, 2018 at 11:00 AM Charles Chen <cc...@google.com> wrote:
>>
>>
>> (sending to the dev@ list thread as this is more relevant here than
>> users@)
>>
>> Will we be using a different / potentially more rigorous process for
>> releasing LTS releases?  Or do we feel that any validations that could
>> possibly be done should already be incorporated into each release?
>>
>> On Mon, Aug 13, 2018 at 10:57 AM Ahmet Altay <al...@google.com> wrote:
>>
>>
>> Update:
>>
>> I sent out an email to user@ to collect their feedback [1]. I will
>> encourage everyone here to collect feedback from the other channels
>> available to you. To facilitate the discussion I drafted my proposal in a
>> PR [2].
>>
>> Ahmet
>>
>> [1] https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c77358345
>> 0767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>> [2] https://github.com/apache/beam-site/pull/537
>>
>> On Fri, Aug 10, 2018 at 5:20 PM, Lukasz Cwik <lc...@google.com> wrote:
>>
>>
>> Thanks, I can see the reasoning for LTS releases based upon some
>> enterprise customers needs.
>>
>> Forgot about the 2.1.1 Python release. Thanks for pointing that out.
>>
>> On Fri, Aug 10, 2018 at 4:47 PM Ahmet Altay <al...@google.com> wrote:
>>
>>
>>
>> On Fri, Aug 10, 2018 at 12:33 PM, Lukasz Cwik <lc...@google.com> wrote:
>>
>>
>> I like the ideas that your proposing but am wondering what value if any
>> do supporting LTS releases add? We maintain semantic versioning and I would
>> expect that most users would be using the latest released version if not
>> the release just before that. There is likely a long tail of users who will
>> use a specific version and are unlikely to ever upgrade.
>>
>>
>>
>> I believe there is a category of enterprise users who would continue to
>> use a specific version as long as they know they can get support for it.
>> This usually stems from the need to have a stable environment. There is
>> also the aspect of validating new product before using. I know some
>> companies have validation cycles longer than 6 weeks. They will still
>> upgrade but they would like to upgrade much less frequently.
>>
>> I was hoping that defining LTS releases will signal these types of users
>> what releases are worth upgrading to if they have a high cost of upgrading.
>>
>> This comes from my anecdotal evidence and I may be wrong.
>>
>>
>>
>> I believe it would be valuable to ask our users what is most important to
>> them with respect to the policy (after we have discussed it a little bit)
>> as well since ultimately our goal is to help our users.
>>
>>
>>
>> I agree with this. Since I am referring to enterprise users primarily I
>> think some of it will require the companies here to collect that feedback.
>>
>>
>> This could then be documented and we could provide guidance to customers
>> as to how to reach out to the group for big bugs. Also note that Apache has
>> a security policy[1] in place which we should direct users to.
>>
>>
>>
>> I think document what could be expected of Beam in terms of support would
>> be very valuable by itself. It will also help us figure out what we could
>> drop. For example in the recent discussion to drop old API docs, there was
>> no clear guidance on which SDKs are still supported and should have their
>> API docs hosted.
>>
>> I think we reference to the Apache security policy on our website. If not
>> I agree, we should add a reference to it.
>>
>>
>>
>> Also, we don't have any experience in patching a release as we haven't
>> yet done one patch version bump. All issues that have been brought up were
>> always fixed in the next minor version bump.
>>
>>
>>
>> I agree. There was the Python 2.1.1 but that is the only example I could
>> remember.
>>
>>
>>
>> 1: http://www.apache.org/security/
>>
>>
>>
>>
>> On Fri, Aug 10, 2018 at 11:50 AM Pablo Estrada <pa...@google.com>
>> wrote:
>>
>>
>> I think this all sounds reasonable, and I think it would be a good story
>> for our users. We don't have much experience with patching releases, but I
>> guess it's a matter of learning and improving over time.
>> -P.
>>
>> On Wed, Aug 8, 2018 at 9:04 PM Ahmet Altay <al...@google.com> wrote:
>>
>>
>> Hi all,
>>
>> I would like us to clarify the life cycle of Beam releases a little bit
>> more for our users. I think we increased the predictability significantly
>> by agreeing to a release cadence and kudos to everyone on that. As a follow
>> up to that I would like to address the following problem:
>>
>> It is unclear for a user of Beam how long an existing version will be
>> supported. And if it will be supported at all, what does that support mean.
>> (This is especially an important problem for users who would like to use
>> stable versions and care less about being on the cutting edge.)
>>
>> Our current state is:
>>
>> - With our agreed release cadence Beam makes 8 releases a year.
>> - We have precedence for patching released versions for major issues.
>> - Patching all existing releases at any point (even patching a year full
>> of 8 releases) will be significant work.
>>
>> With the problem and the information, I have the following proposal to
>> define the life cycle of existing releases.
>>
>> - Define what is a major issue with Beam. (For example this could be high
>> severity security issues and high risk data integrity issues.)
>> - Have a concept of long term support (LTS) releases. Designate every 4th
>> release as an LTS release (~6 months).
>> - Deprecate non-LTS releases the moment any new Beam release is out.
>> Never patch non-LTS releases.
>> - Deprecate LTS releases after a new LTS release comes out. Patch any LTS
>> release within 1 year of its initial release date.
>> - Add the above information to our website.
>>
>> I think this proposal would give clear information to our users about
>> what they can expect from us, and reduce our burden to maintain existing
>> releases.
>>
>> I also would like to state my assumptions:
>>
>> - Releases will happen not because of a policy but because there are
>> volunteers willing to do it. This proposal is only a framework for those
>> volunteers to take action. If Beam does not support its releases, with or
>> without a policy, we will reduce the trust of our users.
>> - After we agreed to have a regular release cadence we started to see
>> improvements towards having regular releases even though we did not
>> perfectly hit 6 weeks mark each time. I do expect the same here: an
>> improvement in the direction of user happiness even if we cannot be perfect.
>>
>> What do you think?
>>
>> Ahmet
>>
>>
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>>
>>
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>>
>>