What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

I agree that LTS releases require more thought. Thank you for raising these
questions. What other open questions do we have related LTS releases?

One way to do this would be to add them to a particular tracking list (e.g.
2.9.0 blocking list) that way we would be ready for an LTS release ahead of
time.

Related to dependencies, I agree with Thomas. If we block on waiting for
dependencies, we may end up taking a long time before making any LTS
release. And the reality of Beam releases right now is that there are no
supported releases today that and in the long term that might hurt user
trust. In my opinion, we need to fix that sooner rather than later.

Ahmet


On Thu, Oct 4, 2018 at 9:27 AM, Ahmet Altay <al...@google.com> wrote:

> Thank you all for the feedback. I will continue with 2.8.0 as a regular
> release and separate the LTS discussion to a new thread.
>
> On Thu, Oct 4, 2018 at 7:58 AM, Thomas Weise <th...@apache.org> wrote:
>
>> Given the feedback so far, we should probably decouple LTS and 2.8.0
>> discussions. In case both converge before 10/10 then fine, but not
>> necessary. I also agree that we should not jump the gun on LTS and minimum
>> 72 hours feedback window for the topic looks appropriate.
>>
>> The issues raised by Tim look like blockers and unless we are confident
>> that they can be addressed as a patch release may warrant to defer LTS? Can
>> we start to tag such JIRAs with an LTS label?
>>
>> On the other hand, I think we could allow for a bit of experimentation
>> error for the first LTS attempt and feed guidelines/policies from
>> learnings/feedback.
>>
>> Dependency updates for LTS: I don't think we should block LTS because
>> there is a newer version of a dependency out there or we should rush
>> updates. If we prioritize stability, then the latest usually isn't the
>> best. In the case of Flink, 1.5.x is probably what most users have at this
>> time and it has seen 4 patch releases. If Flink community continues to
>> support last two minor (X.Y) versions, then 1.5.x support may drop when 1.7
>> comes out, but that does not mean we cannot use it if we were to cut a Beam
>> LTS release today. I generally think that LTS needs to focus more on the
>> stability of Beam itself.
>>
>> Thanks,
>> Thomas
>>
>>
>>
>> On Thu, Oct 4, 2018 at 6:59 AM Alexey Romanenko <ar...@gmail.com>
>> wrote:
>>
>>> Regarding LTS release - I agree that we need to have clear view what
>>> kind of support will be provided for such releases.
>>>
>>> Despite of the concerns mentioned before, I have another one about API
>>> labeled as “@Experimental". I think there are most of IOs, SQL, PCollection
>>> with Schema, etc, labeled with this annotation.
>>> According to definition, such API should be considered as unstable in
>>> terms that it can be changed/removed in next releases.
>>>
>>> So, the question is - how “@Experimental” API affects LTS releases (if
>>> it does)? What kind of support should be provided in this case, especially,
>>> in case if API continued evolving after LTS has been issued? Do we need to
>>> provide a guarantee (another annotation, for example) that API won’t be
>>> changed between two LTS releases?
>>>
>>> And one more related question, which probably deserves another
>>> discussion (or was already discussed) - what is criteria to remove
>>> status “@Experimental” from API? How we decide that API is stable and not
>>> changing anymore?
>>>
>>>
>>> On 4 Oct 2018, at 12:35, Robert Bradshaw <ro...@google.com> wrote:
>>>
>>> +1 to cutting the release.
>>>
>>> I agree that the LTS label requires more discussion. I think it boils
>>> down to the question of whether we are comfortable with encouraging people
>>> to not upgrade to the latest Beam. It probably boils down to creating a
>>> list of (potential) blockers and then going from there. Also, on this note,
>>> I think we should be very conservative in updating dependencies for an LTS
>>> release.
>>>
>>> We could also consider for this release doing an "LTS light" where we
>>> prove the process, gain some experience, but don't promise a full 12 months
>>> of support (say, cutting it to 6 months).
>>>
>>> - Robert
>>>
>>>
>>>
>>>
>>> On Thu, Oct 4, 2018 at 11:25 AM Tim Robertson <ti...@gmail.com>
>>> wrote:
>>>
>>>> I was in the middle of writing something similar when Ismaël posted.
>>>>
>>>> Please do bear in mind that this is an international project and 7hrs
>>>> is not long enough to decide upon something that affects us all.
>>>>
>>>> +1 on cutting 2.8.0 on 10/10 and thank you for pushing it forward
>>>>
>>>> -1 on designating it as LTS:
>>>> While LTS is a statement of expectation in maintenance it also carries
>>>> an element of trust. I propose we should have a separate discussion about
>>>> what we might like to collectively achieve before announcing our first LTS
>>>> edition.
>>>> My concern stems from usability and first impressions - for example:
>>>> - Beam has real issues with HDFS today (BEAM-5036) which I propose as
>>>> blocker for announcing LTS
>>>> - DirectRunner and the inability to run basic pipelines on a few GB of
>>>> data is *really* putting people off our project - we might consider
>>>> exploring that as it affects our "brand"
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Oct 4, 2018 at 11:18 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> Thanks Ahmet for volunteering to do the release, and proposing this as
>>>>> an LTS.
>>>>>
>>>>> I have still some questions on our LTS policies (which may have
>>>>> consequences on the discussed release):
>>>>>
>>>>> What are the expected implications of upgrades in the LTS, e.g. If a
>>>>> connector let’s say Kafka is released using the 1.0 dependency, can it
>>>>> be moved upwards in a LTS to version 2.0 or this will be considered a
>>>>> breaking change and we should only move in minor versions. Will this
>>>>> rule be more relaxed for example for all cloud based dependencies
>>>>> (GCP, AWS) for example if a security issue or correctness/performance
>>>>> improvement?
>>>>>
>>>>> Given that this will last for a year maybe we should raise some of the
>>>>> dependencies to the latest versions. Following the recent discussion
>>>>> on dependencies that cannot be ‘automatically’ updated because of end
>>>>> user consequences, I still think about what we should do with
>>>>> (probably related to the previous paragraph):
>>>>>
>>>>> - Should we move Flink then to 1.6.x considering that 1.5.x won’t be
>>>>> maintained in less than 6 months.
>>>>> - Should we wait and upgrade Spark into version 2.4.0 (which is being
>>>>> voted at this moment but not released but could make sense for a LTS)
>>>>> or just stay in 2.3.x. Spark is less of an issue because it is a
>>>>> provided dep but still worth.
>>>>> - Should we update the IO connectors dependencies to the latest stable
>>>>> versions who aren’t, e.g. Elasticsearch, HBase,
>>>>>
>>>>> Of course the goal is not a last minute rush to do this so it fits in
>>>>> the LTS release, but to see that for LTS we may consider the ‘lasting
>>>>> consequences'.
>>>>>
>>>>> One last comment, next time we discuss a proposal please ensure that
>>>>> we wait at least 24h to reach conclusions or proceed, otherwise this
>>>>> will exclude opinions from people who are not in the right time zone
>>>>> (this is the reason why votes last 72h to ensure that everyone may be
>>>>> aware of what is been voted). This is not a mandatory requirement, but
>>>>> agreeing on a LTS in 7h seems a bit short.
>>>>> On Thu, Oct 4, 2018 at 1:36 AM Ahmet Altay <al...@google.com> wrote:
>>>>> >
>>>>> > Great. I will do the cut on 10/10.
>>>>> >
>>>>> > Let's start by triaging the open issues targeted for 2.8.0 [1]. If
>>>>> you have any issues in this list please resolve them or move to the next
>>>>> release. If you are aware of any critical issues please add to this list.
>>>>> >
>>>>> > Ahmet
>>>>> >
>>>>> > [1] https://issues.apache.org/jira/browse/BEAM-5456?jql=project%
>>>>> 20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20f
>>>>> ixVersion%20%3D%202.8.0%20ORDER%20BY%20priority%20DESC%2C%
>>>>> 20updated%20DESC
>>>>> >
>>>>> > > +1 for the 2.7.0 release schedule. Thanks for volunteering. Do we
>>>>> want a standing owner for the LTS branch (like the Linux kernel has) or
>>>>> will we just take volunteers for each LTS release as they arise?
>>>>> >
>>>>> > We have not thought about this before. IMO, it is better to keep
>>>>> things simple and use the same process (i.e. "we just take volunteers for
>>>>> each LTS release as they arise") for patch releases in the future if/when
>>>>> we happen to need those.
>>>>> >
>>>>> >
>>>>> > On Wed, Oct 3, 2018 at 1:21 PM, Thomas Weise <th...@apache.org> wrote:
>>>>> >>
>>>>> >> +1
>>>>> >>
>>>>> >> On Wed, Oct 3, 2018 at 12:33 PM Ted Yu <yu...@gmail.com> wrote:
>>>>> >>>
>>>>> >>> +1
>>>>> >>>
>>>>> >>> On Wed, Oct 3, 2018 at 9:52 AM Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net> wrote:
>>>>> >>>>
>>>>> >>>> +1
>>>>> >>>>
>>>>> >>>> but we have to be fast in release process. 2.7.0 took more than 1
>>>>> month
>>>>> >>>> to be cut !
>>>>> >>>>
>>>>> >>>> If no blocker, we have to just move forward.
>>>>> >
>>>>> >
>>>>> > +1
>>>>> >
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Regards
>>>>> >>>> JB
>>>>> >>>>
>>>>> >>>> On 03/10/2018 18:25, Ahmet Altay wrote:
>>>>> >>>> > Hi all,
>>>>> >>>> >
>>>>> >>>> > Release cut date for the next release is 10/10 according to
>>>>> Beam release
>>>>> >>>> > calendar [1]. Since the previous release is already mostly
>>>>> wrapped up
>>>>> >>>> > (modulo blog post), I would like to propose starting the next
>>>>> release on
>>>>> >>>> > time (10/10).
>>>>> >>>> >
>>>>> >>>> > Additionally I propose designating this release as the first
>>>>> >>>> > long-term-support (LTS) release [2]. This should have no impact
>>>>> on the
>>>>> >>>> > release process, however it would mean that we commit to patch
>>>>> this
>>>>> >>>> > release for the next 12 months for major issues.
>>>>> >>>> >
>>>>> >>>> > I volunteer to perform this release.
>>>>> >>>> >
>>>>> >>>> > What do you think?
>>>>> >>>> >
>>>>> >>>> > Ahmet
>>>>> >>>> >
>>>>> >>>> > [1] https://calendar.google.com/ca
>>>>> lendar/embed?src=0p73sl034k80oob7seouanigd0%40group.
>>>>> calendar.google.com&ctz=America%2FLos_Angeles
>>>>> >>>> > [2] https://beam.apache.org/community/policies/#releases
>>>>> >>>>
>>>>> >>>> --
>>>>> >>>> Jean-Baptiste Onofré
>>>>> >>>> jbonofre@apache.org
>>>>> >>>> http://blog.nanthrax.net
>>>>> >>>> Talend - http://www.talend.com
>>>>> >
>>>>> >
>>>>>
>>>>
>>>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Thomas Weise <th...@apache.org>]

+1 for the idea of starting with an existing release branch

But I have another question: Are we ready to make LTS releases? Today, do
most/all users (that upgrade) move to the latest release because they
really need new features or other changes that would be incompatible with
an LTS branch or do we see a sizable amount of users that would feel
content with the changes an LTS release can provide them? Did we do a poll
or have sufficient feedback?

Thanks

On Fri, Oct 19, 2018 at 3:24 PM Kenneth Knowles <ke...@apache.org> wrote:

> Pinging this. I think Beam should have a live LTS branch.
>
> I want to suggest a different approach: choose something already released
> to be LTS. This way it has had some usage and we have some confidence there
> are no critical problems.
>
> So how about we make 2.7 the first LTS branch?
>
> Kenn
>
> On Wed, Oct 10, 2018 at 8:18 AM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> some times Ago JB spoke about Beam roadmap. I tend to think this
>> discussion does no make any sense without a clear roadmap. The rational
>> here is that a roadmap will give you the future changes
>> and the potential future versions (we spoke a few times of Beam 3). This
>> does not have to be very factual, a slice of 3 months is ok at that stage.
>> However, if you don't have that,
>> you can say 2.8 will be LTS and we support 2 versions but if 2.9 and 2.10
>> introduce breaking changes, then
>> it leads to a LTS 2.8 no more supported. This is just an example but the
>> ratio "cost(project) / gain(user)" 100% depends the plans for the project,
>> technically there is no blocker to support all releases
>> for life but would any PMC have the will to release beam 0.x now? The
>> point of a LTS for an user is to plan investment, if we are in previous
>> case it does not help IMHO.
>> So maybe grab back the Beam enhancement plans and assign them some fix
>> versions before defining what support model of Beam can be.
>>
>> Just the 2 cts of an outsider.
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>>
>> Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <ch...@google.com>
>> a écrit :
>>
>>>
>>>
>>> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> The simplest thing we can do is just to pin all the deps of the LTS
>>>>> and not move them in any maintenance release if not a strong reason to
>>>>> do so.
>>>>>
>>>>> The next subject is to make maintainers aware of which release will be
>>>>> the LTS in advance so they decide what to do with the dependencies
>>>>> versions. In my previous mail I mentioned all the possible cases that
>>>>> can happen with dependencies and it is clear that one unified policy
>>>>> won’t satisfy every one. So better let the maintainers (who can also
>>>>> ask for user feedback in the ML) to decide about  versions before the
>>>>> release.
>>>>>
>>>>> Alexey’s question is still a really important issue, and has been so
>>>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>>>>> Options are:
>>>>>
>>>>> (1) We keep consistent with Experimental which means that they are
>>>>> still not guarantees (note that this does not mean that they will be
>>>>> broken arbitrarily).
>>>>> (2) We are consistent with the LTS approach which makes them ‘non
>>>>> experimental’ for the LTS so we will guarantee the functionality/API
>>>>> stable.
>>>>>
>>>>> I personally have conflicted opinions I would like to favor (1) but
>>>>> this is not consistent with the whole idea of LTS so probably (2) is
>>>>> wiser.
>>>>>
>>>>
>>>> Yeah, I think (2) is the only viable option.
>>>>
>>>
>>> I think important thing here is that future releases on a  LTS branch
>>> will be patch (bugfix) releases so I don't think we can/should do
>>> API/functionality changes (even if the change is experimental and/or
>>> backwards compatible).
>>>
>>> I think same goes for dependency changes. If the change is to fix a
>>> known bug we can do that in a patch release but if it's to add more
>>> functionality probably that should come in a new minor release instead of a
>>> patch release.
>>>
>>> This is why I think we should be bit careful about "rushed" changes to
>>> major functionalities of Beam going into LTS releases. Just my 2 cents.
>>>
>>> Thanks,
>>> Cham
>>>
>>>
>>>>
>>>>
>>>>> Finally I also worry about Tim’s remarks on performance and quality,
>>>>> even if some of these things effectively can be fixed in a subsequent
>>>>> LTS release. Users will probably prefer a LTS to start with Beam and
>>>>> if the performance/quality of the LTS, this can hurt perception of the
>>>>> project.
>>>>>
>>>>
>>>> Yes, for this reason I think it's important to consider what goes into
>>>> an LTS as well as what happens after. Almost by definition, using an LTS is
>>>> choosing stability over cutting edge features. I'd rather major feature X
>>>> goes in after LTS, and lives in a couple of releases gaining fixes and
>>>> improvements before being released as part of the next LTS, than quickly
>>>> making it into an LTS while brand new (both due to the time period before
>>>> we refine it, and the extra work of porting refinements back).
>>>>
>>>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway,
>>>> waiting for at least x.y.1?
>>>>
>>>> Come to think of it, do we even have to designate releases at LTS at
>>>> the time of release? Perhaps we could instead just keep with our normal
>>>> release cadence, and periodically choose one of them as LTS once we've
>>>> confirmed its stability out in the wild and start backporting to it. (I can
>>>> think of several cons with this approach as well, e.g. generally it's
>>>> easier to backport bugfixes at the time a bugfix is done in master rather
>>>> than swapping in context at a later date, but just thought I'd throw it out
>>>> there.)
>>>>
>>>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > I've seen two mentions that "rushing" is contrary to the goals of
>>>>> LTS. But I wouldn't worry about this. The fact is there is almost nothing
>>>>> you can do to stabilize *prior* to cutting the LTS branch. Stability comes
>>>>> from the branch being long-lived and having multiple releases.
>>>>> >
>>>>> > (I think this is pretty much my version of what JB is saying)
>>>>> >
>>>>> > What a conservative user will do if 2.8.x is declared LTS is to
>>>>> start using the 2.8.x branch after it has had a couple bugfix releases. I
>>>>> don't think it is useful or possible to try for an "extra stable" 2.x.0.
>>>>> >
>>>>> > The arguments about supporting the most widely used versions of
>>>>> runner backends apply regardless of LTS. We should support them if we have
>>>>> the resources to do so.
>>>>> >
>>>>> > Kenn
>>>>> >
>>>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net> wrote:
>>>>> >>>
>>>>> >>> Hi,
>>>>> >>>
>>>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a
>>>>> branch
>>>>> >>> that we guarantee to have fixes on it for a long period of time.
>>>>> >>>
>>>>> >>>
>>>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>>>>> >>> releases on a LTS branch, the only constraint is to avoid to
>>>>> introduce
>>>>> >>> breaking changes.
>>>>> >>
>>>>> >>
>>>>> >> I agree with this perspective. Thank you for sharing this. However
>>>>> the other commenters also had a good point. Requiring users to upgrade
>>>>> their runner version maybe incompatible with the goals of an LTS branch.
>>>>> Ideally the fixes here should be very minimal and targeted.
>>>>> >>
>>>>> >>>
>>>>> >>>
>>>>> >>> So, IMHO, the key part is not release, it's branch.
>>>>> >>>
>>>>> >>> The first thing to decide is the branch.
>>>>> >>>
>>>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x
>>>>> LTS
>>>>> >>> branch. It's a branch where we will cherry-pick some important
>>>>> fixes in
>>>>> >>> the future and where we will cut release. It's the approach I use
>>>>> in
>>>>> >>> other Apache projects (especially Karaf) and it works fine.
>>>>> >>
>>>>> >>
>>>>> >> JB, does Karaf has a documented process that we can re-use? If not
>>>>> could you explain a bit more?
>>>>> >>
>>>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>>>>> release out of that?
>>>>> >>
>>>>> >>>
>>>>> >>>
>>>>> >>> Just my $0.01
>>>>> >>>
>>>>> >>> Regards
>>>>> >>> JB
>>>>> >>>
>>>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>>>>> chamikara@google.com
>>>>> >>> > <ma...@google.com>> wrote:
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>>>>> >>> >     <ma...@google.com>> wrote:
>>>>> >>> >
>>>>> >>> >         I agree that LTS releases require more thought. Thank
>>>>> you for
>>>>> >>> >         raising these questions. What other open questions do we
>>>>> have
>>>>> >>> >         related LTS releases?
>>>>> >>> >
>>>>> >>> >         One way to do this would be to add them to a particular
>>>>> tracking
>>>>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be
>>>>> ready for
>>>>> >>> >         an LTS release ahead of time.
>>>>> >>> >
>>>>> >>> >         Related to dependencies, I agree with Thomas. If we
>>>>> block on
>>>>> >>> >         waiting for dependencies, we may end up taking a long
>>>>> time
>>>>> >>> >         before making any LTS release. And the reality of Beam
>>>>> releases
>>>>> >>> >         right now is that there are no supported releases today
>>>>> that and
>>>>> >>> >         in the long term that might hurt user trust. In my
>>>>> opinion, we
>>>>> >>> >         need to fix that sooner rather than later.
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >     Agree on the idea of focussing on stability instead of
>>>>> feature set
>>>>> >>> >     when it comes to LTS releases. Based on the previous
>>>>> discussion on
>>>>> >>> >     this [1] looks like the intended audience is enterprise
>>>>> customers
>>>>> >>> >     that would like to maintain a stable environment and that
>>>>> usually
>>>>> >>> >     have a long testing cycle before deploying a new version.
>>>>> Seems
>>>>> >>> >     like, rushing in features  or dependency upgrades for an LTS
>>>>> release
>>>>> >>> >     will be counterintuitive for such use cases.
>>>>> >>> >
>>>>> >>> >     On the other hand, I think a part of Ismaël point was that
>>>>> all major
>>>>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc
>>>>> ..)
>>>>> >>> >     should be supportable for the duration of the time Beam LTS
>>>>> release
>>>>> >>> >     is supported (1 year based on [1]). If a major dependency is
>>>>> >>> >     expected to become unsupported within an year, it makes
>>>>> sense to
>>>>> >>> >     upgrade that before the LTS release. I suggest we do this at
>>>>> least
>>>>> >>> >     one release before the planned LTS release (so 2.9.0 if we
>>>>> want to
>>>>> >>> >     make 2.10.0 a LTS release) so that we have all major
>>>>> >>> >     dependency/feature updates battle tested before committing
>>>>> to them.
>>>>> >>> >
>>>>> >>> >
>>>>> >>> > Dependencies are a significant concern. What happens if one of
>>>>> our
>>>>> >>> > dependencies has no version they're going to support for a year
>>>>> hence?
>>>>> >>> > It could happen that there's never a time when all our
>>>>> dependencies will
>>>>> >>> > be supported for at least a year as of any given date. And
>>>>> rushing to
>>>>> >>> > get the "latest" version of a dependency in is often contrary to
>>>>> the
>>>>> >>> > goals of an LTS. (Better to have those depend on the
>>>>> battle-hardened
>>>>> >>> > versions.)
>>>>> >>> >
>>>>> >>> > The long-term goal is probably to avoid pinning specific
>>>>> versions as
>>>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>>>>> (whichever the
>>>>> >>> > user provided), and considered backporting support for 1.7 when
>>>>> it came
>>>>> >>> > out, we'd be in much better shape here. How feasible this is
>>>>> depends on
>>>>> >>> > how backwards-compatible the dependency is.
>>>>> >>> >
>>>>> >>> > On the other hand, forcing a user to upgrade a (user-visible,
>>>>> which in
>>>>> >>> > particular includes runners) dependency seems contrary to the
>>>>> goals of
>>>>> >>> > an LTS. Even if that dependency becomes unsupported.
>>>>> >>> >
>>>>> >>> >
>>>>> >>>
>>>>> >>> --
>>>>> >>> Jean-Baptiste Onofré
>>>>> >>> jbonofre@apache.org
>>>>> >>> http://blog.nanthrax.net
>>>>> >>> Talend - http://www.talend.com
>>>>> >>
>>>>> >>
>>>>>
>>>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

There were no new updates, I will start a vote based on the latest proposal.

On Mon, Nov 5, 2018 at 10:19 AM, Ahmet Altay <al...@google.com> wrote:

> +1 to starting with 2.7 branch and supporting it for 6 months. I think we
> should start the support window of 6 months from the day we agree to do
> this. That way users will at least get the benefit for 6 months after
> learning about LTS status.
>
> It seems like there is a consensus. Should we hold a vote on this?
>
> On Mon, Nov 5, 2018 at 6:46 AM, Robert Bradshaw <ro...@google.com>
> wrote:
>
>> Yes, cutting more patch releases is the goal of the LTS release. We
>> have not yet determined what the threshold is for backporting bugfixes
>> (which, in part, depends on how much work that is) nor how often we'd
>> do a release.
>>
>
> How about we start tagging issues with a fix version 2.7.1 and a do a case
> by case decision. Over time we could write down common patterns that we
> used for deciding what to backport.
>
>
>>
>> On Mon, Nov 5, 2018 at 3:42 PM Chamikara Jayalath <ch...@google.com>
>> wrote:
>> >
>> > +1 for using an existing release.
>> >
>> > Regarding set of issues, I think so far the policy has been that we cut
>> patch releases for major issues such as security fixes or major breakages
>> of functionality (we only did one patch release so far IIRC). Are we going
>> to change this policy ? For example, are we going to cut regular patch
>> releases for supported branch (release-2.7.0) within the supported period
>> that fixes known issues ? My preference is to keep existing policy on this
>> regard.
>> >
>> > Thanks,
>> > Cham
>> >
>> > On Mon, Nov 5, 2018 at 5:12 AM Robert Bradshaw <ro...@google.com>
>> wrote:
>> >>
>> >> Indeed, that's a vary good signal.
>> >>
>> >> I propose we start with the 2.7 branch (which has been out in the wild
>> >> for a bit and seems pretty stable), supported for 6 months (from
>> >> now?). We should gather a list of issues, if any, that merit
>> >> backporting.
>>
> >> On Mon, Nov 5, 2018 at 11:07 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>> >> >
>> >> > The result shows that there is a demand for an LTS release.
>> >> >
>> >> > +1 for using an existing release. How about six months for the
>> initial
>> >> > LTS release? I think it shouldn't be too long for the first one to
>> give
>> >> > us a chance to make changes to the model.
>> >> >
>> >> > -Max
>> >> >
>> >> > On 02.11.18 17:26, Ahmet Altay wrote:
>> >> > > Twitter vote concluded with 52 votes with the following results:
>> >> > > - 52% Stable LTS releases
>> >> > > - 46% Upgrade to latest release
>> >> > > - 2% Keep using older releases
>> >> > >
>> >> > > This reads like another supporting evidence for making LTS
>> releases. In
>> >> > > the light of this, what do you all think about Kenn's proposal of
>> making
>> >> > > existing branch an LTS branch?
>> >> > >
>> >> > > On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com
>> >> > > <ma...@google.com>> wrote:
>> >> > >
>> >> > >
>> >> > >
>> >> > >     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <
>> kenn@apache.org
>> >> > >     <ma...@apache.org>> wrote:
>> >> > >
>> >> > >         Yes, user@ cannot reach new users, really. Twitter might,
>> if we
>> >> > >         have enough of adjacent followers to get it in front of the
>> >> > >         right people. On the other hand, I find testimonials from
>> >> > >         experience convincing in this case.
>> >> > >
>> >> > >
>> >> > >     I agree I am not sure how much additional input we will get
>> from a
>> >> > >     twitter poll. Started one anyway
>> >> > >     (https://twitter.com/ApacheBeam/status/1055598972423684096
>> >> > >     <https://twitter.com/ApacheBeam/status/1055598972423684096>).
>> I used
>> >> > >     Thomas's version as the basis and had to shorten it to fit the
>> >> > >     character limits.
>> >> > >
>> >> > >
>> >> > >         Kenn
>> >> > >
>> >> > >         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <
>> altay@google.com
>> >> > >         <ma...@google.com>> wrote:
>> >> > >
>> >> > >
>> >> > >
>> >> > >             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
>> >> > >             <thw@apache.org <ma...@apache.org>> wrote:
>> >> > >
>> >> > >
>> >> > >
>> >> > >                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
>> >> > >                 <altay@google.com <ma...@google.com>>
>> wrote:
>> >> > >
>> >> > >                     We attempted to collect feedback on the mailing
>> >> > >                     lists but did not get much input. From my
>> experience
>> >> > >                     (mostly based on dataflow) there is a sizeable
>> group
>> >> > >                     of users who are less interested in new
>> features and
>> >> > >                     want a version that is stable, that does not
>> have
>> >> > >                     security issues, major data integrity issues
>> etc. In
>> >> > >                     Beam's existing release model that corresponds
>> to
>> >> > >                     the latest release.
>> >> > >
>> >> > >                     It would help a lot if we can hear the
>> perspectives
>> >> > >                     of other users who are not present here
>> through the
>> >> > >                     developers who work with them.
>> >> > >
>> >> > >
>> >> > >                 Perhaps user@ and Twitter are good ways to reach
>> >> > >                 relevant audience.
>> >> > >
>> >> > >
>> >> > >             We tried user@ before did not get any feedback [1].
>> Polling
>> >> > >             on twitter sounds like a good idea. Unless there is an
>> >> > >             objection, I can start a poll with Thomas's proposed
>> text as
>> >> > >             is on Beam's twitter account.
>> >> > >
>> >> > >             [1]
>> >> > >             https://lists.apache.org/thre
>> ad.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c
>> 7eba@%3Cuser.beam.apache.org%3E
>> >> > >             <https://lists.apache.org/thr
>> ead.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56
>> c7eba@%3Cuser.beam.apache.org%3E>
>> >> > >
>> >> > >
>> >> > >                 A poll could look like this:
>> >> > >
>> >> > >                 The Beam community is considering LTS (Long Term
>> >> > >                 Support) for selected releases. LTS releases would
>> only
>> >> > >                 contain critical bug fixes (security, data
>> integrity
>> >> > >                 etc.) and offer an alternative to upgrading to
>> latest
>> >> > >                 Beam release with new features. Please indicate
>> your
>> >> > >                 preference for Beam upgrades:
>> >> > >
>> >> > >                 1) Always upgrading to the latest release because
>> I need
>> >> > >                 latest features along with bug fixes
>> >> > >                 2) Interested to switch to LTS releases to obtain
>> >> > >                 critical fixes
>> >> > >                 3) Not upgrading (using older release for other
>> reasons)
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>>
>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

+1 to starting with 2.7 branch and supporting it for 6 months. I think we
should start the support window of 6 months from the day we agree to do
this. That way users will at least get the benefit for 6 months after
learning about LTS status.

It seems like there is a consensus. Should we hold a vote on this?

On Mon, Nov 5, 2018 at 6:46 AM, Robert Bradshaw <ro...@google.com> wrote:

> Yes, cutting more patch releases is the goal of the LTS release. We
> have not yet determined what the threshold is for backporting bugfixes
> (which, in part, depends on how much work that is) nor how often we'd
> do a release.
>

How about we start tagging issues with a fix version 2.7.1 and a do a case
by case decision. Over time we could write down common patterns that we
used for deciding what to backport.


>
> On Mon, Nov 5, 2018 at 3:42 PM Chamikara Jayalath <ch...@google.com>
> wrote:
> >
> > +1 for using an existing release.
> >
> > Regarding set of issues, I think so far the policy has been that we cut
> patch releases for major issues such as security fixes or major breakages
> of functionality (we only did one patch release so far IIRC). Are we going
> to change this policy ? For example, are we going to cut regular patch
> releases for supported branch (release-2.7.0) within the supported period
> that fixes known issues ? My preference is to keep existing policy on this
> regard.
> >
> > Thanks,
> > Cham
> >
> > On Mon, Nov 5, 2018 at 5:12 AM Robert Bradshaw <ro...@google.com>
> wrote:
> >>
> >> Indeed, that's a vary good signal.
> >>
> >> I propose we start with the 2.7 branch (which has been out in the wild
> >> for a bit and seems pretty stable), supported for 6 months (from
> >> now?). We should gather a list of issues, if any, that merit
> >> backporting.
>
>> On Mon, Nov 5, 2018 at 11:07 AM Maximilian Michels <mx...@apache.org>
> wrote:
> >> >
> >> > The result shows that there is a demand for an LTS release.
> >> >
> >> > +1 for using an existing release. How about six months for the initial
> >> > LTS release? I think it shouldn't be too long for the first one to
> give
> >> > us a chance to make changes to the model.
> >> >
> >> > -Max
> >> >
> >> > On 02.11.18 17:26, Ahmet Altay wrote:
> >> > > Twitter vote concluded with 52 votes with the following results:
> >> > > - 52% Stable LTS releases
> >> > > - 46% Upgrade to latest release
> >> > > - 2% Keep using older releases
> >> > >
> >> > > This reads like another supporting evidence for making LTS
> releases. In
> >> > > the light of this, what do you all think about Kenn's proposal of
> making
> >> > > existing branch an LTS branch?
> >> > >
> >> > > On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com
> >> > > <ma...@google.com>> wrote:
> >> > >
> >> > >
> >> > >
> >> > >     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <
> kenn@apache.org
> >> > >     <ma...@apache.org>> wrote:
> >> > >
> >> > >         Yes, user@ cannot reach new users, really. Twitter might,
> if we
> >> > >         have enough of adjacent followers to get it in front of the
> >> > >         right people. On the other hand, I find testimonials from
> >> > >         experience convincing in this case.
> >> > >
> >> > >
> >> > >     I agree I am not sure how much additional input we will get
> from a
> >> > >     twitter poll. Started one anyway
> >> > >     (https://twitter.com/ApacheBeam/status/1055598972423684096
> >> > >     <https://twitter.com/ApacheBeam/status/1055598972423684096>).
> I used
> >> > >     Thomas's version as the basis and had to shorten it to fit the
> >> > >     character limits.
> >> > >
> >> > >
> >> > >         Kenn
> >> > >
> >> > >         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <
> altay@google.com
> >> > >         <ma...@google.com>> wrote:
> >> > >
> >> > >
> >> > >
> >> > >             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
> >> > >             <thw@apache.org <ma...@apache.org>> wrote:
> >> > >
> >> > >
> >> > >
> >> > >                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
> >> > >                 <altay@google.com <ma...@google.com>> wrote:
> >> > >
> >> > >                     We attempted to collect feedback on the mailing
> >> > >                     lists but did not get much input. From my
> experience
> >> > >                     (mostly based on dataflow) there is a sizeable
> group
> >> > >                     of users who are less interested in new
> features and
> >> > >                     want a version that is stable, that does not
> have
> >> > >                     security issues, major data integrity issues
> etc. In
> >> > >                     Beam's existing release model that corresponds
> to
> >> > >                     the latest release.
> >> > >
> >> > >                     It would help a lot if we can hear the
> perspectives
> >> > >                     of other users who are not present here through
> the
> >> > >                     developers who work with them.
> >> > >
> >> > >
> >> > >                 Perhaps user@ and Twitter are good ways to reach
> >> > >                 relevant audience.
> >> > >
> >> > >
> >> > >             We tried user@ before did not get any feedback [1].
> Polling
> >> > >             on twitter sounds like a good idea. Unless there is an
> >> > >             objection, I can start a poll with Thomas's proposed
> text as
> >> > >             is on Beam's twitter account.
> >> > >
> >> > >             [1]
> >> > >             https://lists.apache.org/thread.html/
> 7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%
> 3Cuser.beam.apache.org%3E
> >> > >             <https://lists.apache.org/thread.html/
> 7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%
> 3Cuser.beam.apache.org%3E>
> >> > >
> >> > >
> >> > >                 A poll could look like this:
> >> > >
> >> > >                 The Beam community is considering LTS (Long Term
> >> > >                 Support) for selected releases. LTS releases would
> only
> >> > >                 contain critical bug fixes (security, data integrity
> >> > >                 etc.) and offer an alternative to upgrading to
> latest
> >> > >                 Beam release with new features. Please indicate your
> >> > >                 preference for Beam upgrades:
> >> > >
> >> > >                 1) Always upgrading to the latest release because I
> need
> >> > >                 latest features along with bug fixes
> >> > >                 2) Interested to switch to LTS releases to obtain
> >> > >                 critical fixes
> >> > >                 3) Not upgrading (using older release for other
> reasons)
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Robert Bradshaw <ro...@google.com>]

Yes, cutting more patch releases is the goal of the LTS release. We
have not yet determined what the threshold is for backporting bugfixes
(which, in part, depends on how much work that is) nor how often we'd
do a release.

On Mon, Nov 5, 2018 at 3:42 PM Chamikara Jayalath <ch...@google.com> wrote:
>
> +1 for using an existing release.
>
> Regarding set of issues, I think so far the policy has been that we cut patch releases for major issues such as security fixes or major breakages of functionality (we only did one patch release so far IIRC). Are we going to change this policy ? For example, are we going to cut regular patch releases for supported branch (release-2.7.0) within the supported period that fixes known issues ? My preference is to keep existing policy on this regard.
>
> Thanks,
> Cham
>
> On Mon, Nov 5, 2018 at 5:12 AM Robert Bradshaw <ro...@google.com> wrote:
>>
>> Indeed, that's a vary good signal.
>>
>> I propose we start with the 2.7 branch (which has been out in the wild
>> for a bit and seems pretty stable), supported for 6 months (from
>> now?). We should gather a list of issues, if any, that merit
>> backporting.
>> On Mon, Nov 5, 2018 at 11:07 AM Maximilian Michels <mx...@apache.org> wrote:
>> >
>> > The result shows that there is a demand for an LTS release.
>> >
>> > +1 for using an existing release. How about six months for the initial
>> > LTS release? I think it shouldn't be too long for the first one to give
>> > us a chance to make changes to the model.
>> >
>> > -Max
>> >
>> > On 02.11.18 17:26, Ahmet Altay wrote:
>> > > Twitter vote concluded with 52 votes with the following results:
>> > > - 52% Stable LTS releases
>> > > - 46% Upgrade to latest release
>> > > - 2% Keep using older releases
>> > >
>> > > This reads like another supporting evidence for making LTS releases. In
>> > > the light of this, what do you all think about Kenn's proposal of making
>> > > existing branch an LTS branch?
>> > >
>> > > On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com
>> > > <ma...@google.com>> wrote:
>> > >
>> > >
>> > >
>> > >     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <kenn@apache.org
>> > >     <ma...@apache.org>> wrote:
>> > >
>> > >         Yes, user@ cannot reach new users, really. Twitter might, if we
>> > >         have enough of adjacent followers to get it in front of the
>> > >         right people. On the other hand, I find testimonials from
>> > >         experience convincing in this case.
>> > >
>> > >
>> > >     I agree I am not sure how much additional input we will get from a
>> > >     twitter poll. Started one anyway
>> > >     (https://twitter.com/ApacheBeam/status/1055598972423684096
>> > >     <https://twitter.com/ApacheBeam/status/1055598972423684096>). I used
>> > >     Thomas's version as the basis and had to shorten it to fit the
>> > >     character limits.
>> > >
>> > >
>> > >         Kenn
>> > >
>> > >         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <altay@google.com
>> > >         <ma...@google.com>> wrote:
>> > >
>> > >
>> > >
>> > >             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
>> > >             <thw@apache.org <ma...@apache.org>> wrote:
>> > >
>> > >
>> > >
>> > >                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
>> > >                 <altay@google.com <ma...@google.com>> wrote:
>> > >
>> > >                     We attempted to collect feedback on the mailing
>> > >                     lists but did not get much input. From my experience
>> > >                     (mostly based on dataflow) there is a sizeable group
>> > >                     of users who are less interested in new features and
>> > >                     want a version that is stable, that does not have
>> > >                     security issues, major data integrity issues etc. In
>> > >                     Beam's existing release model that corresponds to
>> > >                     the latest release.
>> > >
>> > >                     It would help a lot if we can hear the perspectives
>> > >                     of other users who are not present here through the
>> > >                     developers who work with them.
>> > >
>> > >
>> > >                 Perhaps user@ and Twitter are good ways to reach
>> > >                 relevant audience.
>> > >
>> > >
>> > >             We tried user@ before did not get any feedback [1]. Polling
>> > >             on twitter sounds like a good idea. Unless there is an
>> > >             objection, I can start a poll with Thomas's proposed text as
>> > >             is on Beam's twitter account.
>> > >
>> > >             [1]
>> > >             https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>> > >             <https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E>
>> > >
>> > >
>> > >                 A poll could look like this:
>> > >
>> > >                 The Beam community is considering LTS (Long Term
>> > >                 Support) for selected releases. LTS releases would only
>> > >                 contain critical bug fixes (security, data integrity
>> > >                 etc.) and offer an alternative to upgrading to latest
>> > >                 Beam release with new features. Please indicate your
>> > >                 preference for Beam upgrades:
>> > >
>> > >                 1) Always upgrading to the latest release because I need
>> > >                 latest features along with bug fixes
>> > >                 2) Interested to switch to LTS releases to obtain
>> > >                 critical fixes
>> > >                 3) Not upgrading (using older release for other reasons)
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Chamikara Jayalath <ch...@google.com>]

+1 for using an existing release.

Regarding set of issues, I think so far the policy has been that we cut
patch releases for major issues such as security fixes or major breakages
of functionality (we only did one patch release so far IIRC). Are we going
to change this policy ? For example, are we going to cut regular patch
releases for supported branch (release-2.7.0) within the supported period
that fixes known issues ? My preference is to keep existing policy on this
regard.

Thanks,
Cham

On Mon, Nov 5, 2018 at 5:12 AM Robert Bradshaw <ro...@google.com> wrote:

> Indeed, that's a vary good signal.
>
> I propose we start with the 2.7 branch (which has been out in the wild
> for a bit and seems pretty stable), supported for 6 months (from
> now?). We should gather a list of issues, if any, that merit
> backporting.
> On Mon, Nov 5, 2018 at 11:07 AM Maximilian Michels <mx...@apache.org> wrote:
> >
> > The result shows that there is a demand for an LTS release.
> >
> > +1 for using an existing release. How about six months for the initial
> > LTS release? I think it shouldn't be too long for the first one to give
> > us a chance to make changes to the model.
> >
> > -Max
> >
> > On 02.11.18 17:26, Ahmet Altay wrote:
> > > Twitter vote concluded with 52 votes with the following results:
> > > - 52% Stable LTS releases
> > > - 46% Upgrade to latest release
> > > - 2% Keep using older releases
> > >
> > > This reads like another supporting evidence for making LTS releases. In
> > > the light of this, what do you all think about Kenn's proposal of
> making
> > > existing branch an LTS branch?
> > >
> > > On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com
> > > <ma...@google.com>> wrote:
> > >
> > >
> > >
> > >     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <kenn@apache.org
> > >     <ma...@apache.org>> wrote:
> > >
> > >         Yes, user@ cannot reach new users, really. Twitter might, if
> we
> > >         have enough of adjacent followers to get it in front of the
> > >         right people. On the other hand, I find testimonials from
> > >         experience convincing in this case.
> > >
> > >
> > >     I agree I am not sure how much additional input we will get from a
> > >     twitter poll. Started one anyway
> > >     (https://twitter.com/ApacheBeam/status/1055598972423684096
> > >     <https://twitter.com/ApacheBeam/status/1055598972423684096>). I
> used
> > >     Thomas's version as the basis and had to shorten it to fit the
> > >     character limits.
> > >
> > >
> > >         Kenn
> > >
> > >         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <altay@google.com
> > >         <ma...@google.com>> wrote:
> > >
> > >
> > >
> > >             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
> > >             <thw@apache.org <ma...@apache.org>> wrote:
> > >
> > >
> > >
> > >                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
> > >                 <altay@google.com <ma...@google.com>> wrote:
> > >
> > >                     We attempted to collect feedback on the mailing
> > >                     lists but did not get much input. From my
> experience
> > >                     (mostly based on dataflow) there is a sizeable
> group
> > >                     of users who are less interested in new features
> and
> > >                     want a version that is stable, that does not have
> > >                     security issues, major data integrity issues etc.
> In
> > >                     Beam's existing release model that corresponds to
> > >                     the latest release.
> > >
> > >                     It would help a lot if we can hear the perspectives
> > >                     of other users who are not present here through the
> > >                     developers who work with them.
> > >
> > >
> > >                 Perhaps user@ and Twitter are good ways to reach
> > >                 relevant audience.
> > >
> > >
> > >             We tried user@ before did not get any feedback [1].
> Polling
> > >             on twitter sounds like a good idea. Unless there is an
> > >             objection, I can start a poll with Thomas's proposed text
> as
> > >             is on Beam's twitter account.
> > >
> > >             [1]
> > >
> https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
> > >             <
> https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
> >
> > >
> > >
> > >                 A poll could look like this:
> > >
> > >                 The Beam community is considering LTS (Long Term
> > >                 Support) for selected releases. LTS releases would only
> > >                 contain critical bug fixes (security, data integrity
> > >                 etc.) and offer an alternative to upgrading to latest
> > >                 Beam release with new features. Please indicate your
> > >                 preference for Beam upgrades:
> > >
> > >                 1) Always upgrading to the latest release because I
> need
> > >                 latest features along with bug fixes
> > >                 2) Interested to switch to LTS releases to obtain
> > >                 critical fixes
> > >                 3) Not upgrading (using older release for other
> reasons)
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Robert Bradshaw <ro...@google.com>]

Indeed, that's a vary good signal.

I propose we start with the 2.7 branch (which has been out in the wild
for a bit and seems pretty stable), supported for 6 months (from
now?). We should gather a list of issues, if any, that merit
backporting.
On Mon, Nov 5, 2018 at 11:07 AM Maximilian Michels <mx...@apache.org> wrote:
>
> The result shows that there is a demand for an LTS release.
>
> +1 for using an existing release. How about six months for the initial
> LTS release? I think it shouldn't be too long for the first one to give
> us a chance to make changes to the model.
>
> -Max
>
> On 02.11.18 17:26, Ahmet Altay wrote:
> > Twitter vote concluded with 52 votes with the following results:
> > - 52% Stable LTS releases
> > - 46% Upgrade to latest release
> > - 2% Keep using older releases
> >
> > This reads like another supporting evidence for making LTS releases. In
> > the light of this, what do you all think about Kenn's proposal of making
> > existing branch an LTS branch?
> >
> > On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com
> > <ma...@google.com>> wrote:
> >
> >
> >
> >     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <kenn@apache.org
> >     <ma...@apache.org>> wrote:
> >
> >         Yes, user@ cannot reach new users, really. Twitter might, if we
> >         have enough of adjacent followers to get it in front of the
> >         right people. On the other hand, I find testimonials from
> >         experience convincing in this case.
> >
> >
> >     I agree I am not sure how much additional input we will get from a
> >     twitter poll. Started one anyway
> >     (https://twitter.com/ApacheBeam/status/1055598972423684096
> >     <https://twitter.com/ApacheBeam/status/1055598972423684096>). I used
> >     Thomas's version as the basis and had to shorten it to fit the
> >     character limits.
> >
> >
> >         Kenn
> >
> >         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <altay@google.com
> >         <ma...@google.com>> wrote:
> >
> >
> >
> >             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
> >             <thw@apache.org <ma...@apache.org>> wrote:
> >
> >
> >
> >                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
> >                 <altay@google.com <ma...@google.com>> wrote:
> >
> >                     We attempted to collect feedback on the mailing
> >                     lists but did not get much input. From my experience
> >                     (mostly based on dataflow) there is a sizeable group
> >                     of users who are less interested in new features and
> >                     want a version that is stable, that does not have
> >                     security issues, major data integrity issues etc. In
> >                     Beam's existing release model that corresponds to
> >                     the latest release.
> >
> >                     It would help a lot if we can hear the perspectives
> >                     of other users who are not present here through the
> >                     developers who work with them.
> >
> >
> >                 Perhaps user@ and Twitter are good ways to reach
> >                 relevant audience.
> >
> >
> >             We tried user@ before did not get any feedback [1]. Polling
> >             on twitter sounds like a good idea. Unless there is an
> >             objection, I can start a poll with Thomas's proposed text as
> >             is on Beam's twitter account.
> >
> >             [1]
> >             https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
> >             <https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E>
> >
> >
> >                 A poll could look like this:
> >
> >                 The Beam community is considering LTS (Long Term
> >                 Support) for selected releases. LTS releases would only
> >                 contain critical bug fixes (security, data integrity
> >                 etc.) and offer an alternative to upgrading to latest
> >                 Beam release with new features. Please indicate your
> >                 preference for Beam upgrades:
> >
> >                 1) Always upgrading to the latest release because I need
> >                 latest features along with bug fixes
> >                 2) Interested to switch to LTS releases to obtain
> >                 critical fixes
> >                 3) Not upgrading (using older release for other reasons)
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Maximilian Michels <mx...@apache.org>]

The result shows that there is a demand for an LTS release.

+1 for using an existing release. How about six months for the initial 
LTS release? I think it shouldn't be too long for the first one to give 
us a chance to make changes to the model.

-Max

On 02.11.18 17:26, Ahmet Altay wrote:
> Twitter vote concluded with 52 votes with the following results:
> - 52% Stable LTS releases
> - 46% Upgrade to latest release
> - 2% Keep using older releases
> 
> This reads like another supporting evidence for making LTS releases. In 
> the light of this, what do you all think about Kenn's proposal of making 
> existing branch an LTS branch?
> 
> On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <altay@google.com 
> <ma...@google.com>> wrote:
> 
> 
> 
>     On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <kenn@apache.org
>     <ma...@apache.org>> wrote:
> 
>         Yes, user@ cannot reach new users, really. Twitter might, if we
>         have enough of adjacent followers to get it in front of the
>         right people. On the other hand, I find testimonials from
>         experience convincing in this case.
> 
> 
>     I agree I am not sure how much additional input we will get from a
>     twitter poll. Started one anyway
>     (https://twitter.com/ApacheBeam/status/1055598972423684096
>     <https://twitter.com/ApacheBeam/status/1055598972423684096>). I used
>     Thomas's version as the basis and had to shorten it to fit the
>     character limits.
> 
> 
>         Kenn
> 
>         On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <altay@google.com
>         <ma...@google.com>> wrote:
> 
> 
> 
>             On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise
>             <thw@apache.org <ma...@apache.org>> wrote:
> 
> 
> 
>                 On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay
>                 <altay@google.com <ma...@google.com>> wrote:
> 
>                     We attempted to collect feedback on the mailing
>                     lists but did not get much input. From my experience
>                     (mostly based on dataflow) there is a sizeable group
>                     of users who are less interested in new features and
>                     want a version that is stable, that does not have
>                     security issues, major data integrity issues etc. In
>                     Beam's existing release model that corresponds to
>                     the latest release.
> 
>                     It would help a lot if we can hear the perspectives
>                     of other users who are not present here through the
>                     developers who work with them.
> 
> 
>                 Perhaps user@ and Twitter are good ways to reach
>                 relevant audience.
> 
> 
>             We tried user@ before did not get any feedback [1]. Polling
>             on twitter sounds like a good idea. Unless there is an
>             objection, I can start a poll with Thomas's proposed text as
>             is on Beam's twitter account.
> 
>             [1]
>             https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>             <https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E>
> 
> 
>                 A poll could look like this:
> 
>                 The Beam community is considering LTS (Long Term
>                 Support) for selected releases. LTS releases would only
>                 contain critical bug fixes (security, data integrity
>                 etc.) and offer an alternative to upgrading to latest
>                 Beam release with new features. Please indicate your
>                 preference for Beam upgrades:
> 
>                 1) Always upgrading to the latest release because I need
>                 latest features along with bug fixes
>                 2) Interested to switch to LTS releases to obtain
>                 critical fixes
>                 3) Not upgrading (using older release for other reasons)
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

Twitter vote concluded with 52 votes with the following results:
- 52% Stable LTS releases
- 46% Upgrade to latest release
- 2% Keep using older releases

This reads like another supporting evidence for making LTS releases. In the
light of this, what do you all think about Kenn's proposal of making
existing branch an LTS branch?

On Thu, Oct 25, 2018 at 4:18 PM, Ahmet Altay <al...@google.com> wrote:

>
>
> On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <ke...@apache.org> wrote:
>
>> Yes, user@ cannot reach new users, really. Twitter might, if we have
>> enough of adjacent followers to get it in front of the right people. On the
>> other hand, I find testimonials from experience convincing in this case.
>>
>
> I agree I am not sure how much additional input we will get from a twitter
> poll. Started one anyway (https://twitter.com/ApacheBeam/status/
> 1055598972423684096). I used Thomas's version as the basis and had to
> shorten it to fit the character limits.
>
>
>>
>> Kenn
>>
>> On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise <th...@apache.org> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> We attempted to collect feedback on the mailing lists but did not get
>>>>> much input. From my experience (mostly based on dataflow) there is a
>>>>> sizeable group of users who are less interested in new features and want a
>>>>> version that is stable, that does not have security issues, major data
>>>>> integrity issues etc. In Beam's existing release model that corresponds to
>>>>> the latest release.
>>>>>
>>>>> It would help a lot if we can hear the perspectives of other users who
>>>>> are not present here through the developers who work with them.
>>>>>
>>>>
>>>> Perhaps user@ and Twitter are good ways to reach relevant audience.
>>>>
>>>
>>> We tried user@ before did not get any feedback [1]. Polling on twitter
>>> sounds like a good idea. Unless there is an objection, I can start a poll
>>> with Thomas's proposed text as is on Beam's twitter account.
>>>
>>> [1] https://lists.apache.org/thread.html/7d890d6ed221c722a95
>>> d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>>>
>>>
>>>>
>>>> A poll could look like this:
>>>>
>>>> The Beam community is considering LTS (Long Term Support) for selected
>>>> releases. LTS releases would only contain critical bug fixes (security,
>>>> data integrity etc.) and offer an alternative to upgrading to latest Beam
>>>> release with new features. Please indicate your preference for Beam
>>>> upgrades:
>>>>
>>>> 1) Always upgrading to the latest release because I need latest
>>>> features along with bug fixes
>>>> 2) Interested to switch to LTS releases to obtain critical fixes
>>>> 3) Not upgrading (using older release for other reasons)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

On Tue, Oct 23, 2018 at 3:03 PM, Kenneth Knowles <ke...@apache.org> wrote:

> Yes, user@ cannot reach new users, really. Twitter might, if we have
> enough of adjacent followers to get it in front of the right people. On the
> other hand, I find testimonials from experience convincing in this case.
>

I agree I am not sure how much additional input we will get from a twitter
poll. Started one anyway (
https://twitter.com/ApacheBeam/status/1055598972423684096). I used Thomas's
version as the basis and had to shorten it to fit the character limits.


>
> Kenn
>
> On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise <th...@apache.org> wrote:
>>
>>>
>>>
>>> On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> We attempted to collect feedback on the mailing lists but did not get
>>>> much input. From my experience (mostly based on dataflow) there is a
>>>> sizeable group of users who are less interested in new features and want a
>>>> version that is stable, that does not have security issues, major data
>>>> integrity issues etc. In Beam's existing release model that corresponds to
>>>> the latest release.
>>>>
>>>> It would help a lot if we can hear the perspectives of other users who
>>>> are not present here through the developers who work with them.
>>>>
>>>
>>> Perhaps user@ and Twitter are good ways to reach relevant audience.
>>>
>>
>> We tried user@ before did not get any feedback [1]. Polling on twitter
>> sounds like a good idea. Unless there is an objection, I can start a poll
>> with Thomas's proposed text as is on Beam's twitter account.
>>
>> [1] https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c77358345
>> 0767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>>
>>
>>>
>>> A poll could look like this:
>>>
>>> The Beam community is considering LTS (Long Term Support) for selected
>>> releases. LTS releases would only contain critical bug fixes (security,
>>> data integrity etc.) and offer an alternative to upgrading to latest Beam
>>> release with new features. Please indicate your preference for Beam
>>> upgrades:
>>>
>>> 1) Always upgrading to the latest release because I need latest features
>>> along with bug fixes
>>> 2) Interested to switch to LTS releases to obtain critical fixes
>>> 3) Not upgrading (using older release for other reasons)
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Kenneth Knowles <ke...@apache.org>]

Yes, user@ cannot reach new users, really. Twitter might, if we have enough
of adjacent followers to get it in front of the right people. On the other
hand, I find testimonials from experience convincing in this case.

Kenn

On Tue, Oct 23, 2018 at 2:59 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise <th...@apache.org> wrote:
>
>>
>>
>> On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> We attempted to collect feedback on the mailing lists but did not get
>>> much input. From my experience (mostly based on dataflow) there is a
>>> sizeable group of users who are less interested in new features and want a
>>> version that is stable, that does not have security issues, major data
>>> integrity issues etc. In Beam's existing release model that corresponds to
>>> the latest release.
>>>
>>> It would help a lot if we can hear the perspectives of other users who
>>> are not present here through the developers who work with them.
>>>
>>
>> Perhaps user@ and Twitter are good ways to reach relevant audience.
>>
>
> We tried user@ before did not get any feedback [1]. Polling on twitter
> sounds like a good idea. Unless there is an objection, I can start a poll
> with Thomas's proposed text as is on Beam's twitter account.
>
> [1]
> https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E
>
>
>>
>> A poll could look like this:
>>
>> The Beam community is considering LTS (Long Term Support) for selected
>> releases. LTS releases would only contain critical bug fixes (security,
>> data integrity etc.) and offer an alternative to upgrading to latest Beam
>> release with new features. Please indicate your preference for Beam
>> upgrades:
>>
>> 1) Always upgrading to the latest release because I need latest features
>> along with bug fixes
>> 2) Interested to switch to LTS releases to obtain critical fixes
>> 3) Not upgrading (using older release for other reasons)
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

On Tue, Oct 23, 2018 at 9:16 AM, Thomas Weise <th...@apache.org> wrote:

>
>
> On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay <al...@google.com> wrote:
>
>> We attempted to collect feedback on the mailing lists but did not get
>> much input. From my experience (mostly based on dataflow) there is a
>> sizeable group of users who are less interested in new features and want a
>> version that is stable, that does not have security issues, major data
>> integrity issues etc. In Beam's existing release model that corresponds to
>> the latest release.
>>
>> It would help a lot if we can hear the perspectives of other users who
>> are not present here through the developers who work with them.
>>
>
> Perhaps user@ and Twitter are good ways to reach relevant audience.
>

We tried user@ before did not get any feedback [1]. Polling on twitter
sounds like a good idea. Unless there is an objection, I can start a poll
with Thomas's proposed text as is on Beam's twitter account.

[1]
https://lists.apache.org/thread.html/7d890d6ed221c722a95d9c773583450767b79ee0c0c78f48a56c7eba@%3Cuser.beam.apache.org%3E


>
> A poll could look like this:
>
> The Beam community is considering LTS (Long Term Support) for selected
> releases. LTS releases would only contain critical bug fixes (security,
> data integrity etc.) and offer an alternative to upgrading to latest Beam
> release with new features. Please indicate your preference for Beam
> upgrades:
>
> 1) Always upgrading to the latest release because I need latest features
> along with bug fixes
> 2) Interested to switch to LTS releases to obtain critical fixes
> 3) Not upgrading (using older release for other reasons)
>
>
>
>
>
>
>
>
>
>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Thomas Weise <th...@apache.org>]

On Mon, Oct 22, 2018 at 2:42 PM Ahmet Altay <al...@google.com> wrote:

> We attempted to collect feedback on the mailing lists but did not get much
> input. From my experience (mostly based on dataflow) there is a sizeable
> group of users who are less interested in new features and want a version
> that is stable, that does not have security issues, major data integrity
> issues etc. In Beam's existing release model that corresponds to the latest
> release.
>
> It would help a lot if we can hear the perspectives of other users who are
> not present here through the developers who work with them.
>

Perhaps user@ and Twitter are good ways to reach relevant audience.

A poll could look like this:

The Beam community is considering LTS (Long Term Support) for selected
releases. LTS releases would only contain critical bug fixes (security,
data integrity etc.) and offer an alternative to upgrading to latest Beam
release with new features. Please indicate your preference for Beam
upgrades:

1) Always upgrading to the latest release because I need latest features
along with bug fixes
2) Interested to switch to LTS releases to obtain critical fixes
3) Not upgrading (using older release for other reasons)

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

On Mon, Oct 22, 2018 at 2:11 AM, Robert Bradshaw <ro...@google.com>
wrote:

> On Sat, Oct 20, 2018 at 12:24 AM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Pinging this. I think Beam should have a live LTS branch.
>>
>> I want to suggest a different approach: choose something already released
>> to be LTS. This way it has had some usage and we have some confidence there
>> are no critical problems.
>>
>
> +1. This is what I suggested as well.
>

I agree, great suggestion to base on a known good release.


>
>
>> So how about we make 2.7 the first LTS branch?
>>
>
> I would be fine with that, however, I think our first stab at an LTS
> branch should not promise a full year of support, more like a test run.
>

How could we cleanly communicate this to the users if we go with the test
run approach? My worry is that if we label it as LTS people will treat is
as it is supported. We could choose to not label it anything and still try
to provide patches as a test run. However, that would not be very helpful
because user would not necessarily stay on 2.7 unless they know that we
have an intention of keeping the branch patched for a while.

How about marking it LTS but with a special note on the release notes with
a specifically shorter time period? (e.g. 2.7.0 release date + 6 months as
the test run period.)


> +1 to Thomas' point that concrete user input/feedback would be useful in
> planning here.
>

> But I have another question: Are we ready to make LTS releases? Today, do
most/all users (that upgrade) move to the latest release because they
really need new features or other changes that would be incompatible with
an LTS branch or do we see a sizable amount of users that would feel
content with the changes an LTS release can provide them? Did we do a poll
or have sufficient feedback?

We attempted to collect feedback on the mailing lists but did not get much
input. From my experience (mostly based on dataflow) there is a sizeable
group of users who are less interested in new features and want a version
that is stable, that does not have security issues, major data integrity
issues etc. In Beam's existing release model that corresponds to the latest
release.

It would help a lot if we can hear the perspectives of other users who are
not present here through the developers who work with them.


>
>> Kenn
>>
>> On Wed, Oct 10, 2018 at 8:18 AM Romain Manni-Bucau <rm...@gmail.com>
>> wrote:
>>
>>> some times Ago JB spoke about Beam roadmap. I tend to think this
>>> discussion does no make any sense without a clear roadmap. The rational
>>> here is that a roadmap will give you the future changes
>>> and the potential future versions (we spoke a few times of Beam 3). This
>>> does not have to be very factual, a slice of 3 months is ok at that stage.
>>> However, if you don't have that,
>>> you can say 2.8 will be LTS and we support 2 versions but if 2.9 and
>>> 2.10 introduce breaking changes, then
>>> it leads to a LTS 2.8 no more supported. This is just an example but the
>>> ratio "cost(project) / gain(user)" 100% depends the plans for the project,
>>> technically there is no blocker to support all releases
>>> for life but would any PMC have the will to release beam 0.x now? The
>>> point of a LTS for an user is to plan investment, if we are in previous
>>> case it does not help IMHO.
>>> So maybe grab back the Beam enhancement plans and assign them some fix
>>> versions before defining what support model of Beam can be.
>>>
>>> Just the 2 cts of an outsider.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>>
>>>
>>> Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <ch...@google.com>
>>> a écrit :
>>>
>>>>
>>>>
>>>> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> The simplest thing we can do is just to pin all the deps of the LTS
>>>>>> and not move them in any maintenance release if not a strong reason to
>>>>>> do so.
>>>>>>
>>>>>> The next subject is to make maintainers aware of which release will be
>>>>>> the LTS in advance so they decide what to do with the dependencies
>>>>>> versions. In my previous mail I mentioned all the possible cases that
>>>>>> can happen with dependencies and it is clear that one unified policy
>>>>>> won’t satisfy every one. So better let the maintainers (who can also
>>>>>> ask for user feedback in the ML) to decide about  versions before the
>>>>>> release.
>>>>>>
>>>>>> Alexey’s question is still a really important issue, and has been so
>>>>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>>>>>> Options are:
>>>>>>
>>>>>> (1) We keep consistent with Experimental which means that they are
>>>>>> still not guarantees (note that this does not mean that they will be
>>>>>> broken arbitrarily).
>>>>>> (2) We are consistent with the LTS approach which makes them ‘non
>>>>>> experimental’ for the LTS so we will guarantee the functionality/API
>>>>>> stable.
>>>>>>
>>>>>> I personally have conflicted opinions I would like to favor (1) but
>>>>>> this is not consistent with the whole idea of LTS so probably (2) is
>>>>>> wiser.
>>>>>>
>>>>>
>>>>> Yeah, I think (2) is the only viable option.
>>>>>
>>>>
>>>> I think important thing here is that future releases on a  LTS branch
>>>> will be patch (bugfix) releases so I don't think we can/should do
>>>> API/functionality changes (even if the change is experimental and/or
>>>> backwards compatible).
>>>>
>>>> I think same goes for dependency changes. If the change is to fix a
>>>> known bug we can do that in a patch release but if it's to add more
>>>> functionality probably that should come in a new minor release instead of a
>>>> patch release.
>>>>
>>>> This is why I think we should be bit careful about "rushed" changes to
>>>> major functionalities of Beam going into LTS releases. Just my 2 cents.
>>>>
>>>> Thanks,
>>>> Cham
>>>>
>>>>
>>>>>
>>>>>
>>>>>> Finally I also worry about Tim’s remarks on performance and quality,
>>>>>> even if some of these things effectively can be fixed in a subsequent
>>>>>> LTS release. Users will probably prefer a LTS to start with Beam and
>>>>>> if the performance/quality of the LTS, this can hurt perception of the
>>>>>> project.
>>>>>>
>>>>>
>>>>> Yes, for this reason I think it's important to consider what goes into
>>>>> an LTS as well as what happens after. Almost by definition, using an LTS is
>>>>> choosing stability over cutting edge features. I'd rather major feature X
>>>>> goes in after LTS, and lives in a couple of releases gaining fixes and
>>>>> improvements before being released as part of the next LTS, than quickly
>>>>> making it into an LTS while brand new (both due to the time period before
>>>>> we refine it, and the extra work of porting refinements back).
>>>>>
>>>>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway,
>>>>> waiting for at least x.y.1?
>>>>>
>>>>> Come to think of it, do we even have to designate releases at LTS at
>>>>> the time of release? Perhaps we could instead just keep with our normal
>>>>> release cadence, and periodically choose one of them as LTS once we've
>>>>> confirmed its stability out in the wild and start backporting to it. (I can
>>>>> think of several cons with this approach as well, e.g. generally it's
>>>>> easier to backport bugfixes at the time a bugfix is done in master rather
>>>>> than swapping in context at a later date, but just thought I'd throw it out
>>>>> there.)
>>>>>
>>>>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > I've seen two mentions that "rushing" is contrary to the goals of
>>>>>> LTS. But I wouldn't worry about this. The fact is there is almost nothing
>>>>>> you can do to stabilize *prior* to cutting the LTS branch. Stability comes
>>>>>> from the branch being long-lived and having multiple releases.
>>>>>> >
>>>>>> > (I think this is pretty much my version of what JB is saying)
>>>>>> >
>>>>>> > What a conservative user will do if 2.8.x is declared LTS is to
>>>>>> start using the 2.8.x branch after it has had a couple bugfix releases. I
>>>>>> don't think it is useful or possible to try for an "extra stable" 2.x.0.
>>>>>> >
>>>>>> > The arguments about supporting the most widely used versions of
>>>>>> runner backends apply regardless of LTS. We should support them if we have
>>>>>> the resources to do so.
>>>>>> >
>>>>>> > Kenn
>>>>>> >
>>>>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net> wrote:
>>>>>> >>>
>>>>>> >>> Hi,
>>>>>> >>>
>>>>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a
>>>>>> branch
>>>>>> >>> that we guarantee to have fixes on it for a long period of time.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>>>>>> >>> releases on a LTS branch, the only constraint is to avoid to
>>>>>> introduce
>>>>>> >>> breaking changes.
>>>>>> >>
>>>>>> >>
>>>>>> >> I agree with this perspective. Thank you for sharing this. However
>>>>>> the other commenters also had a good point. Requiring users to upgrade
>>>>>> their runner version maybe incompatible with the goals of an LTS branch.
>>>>>> Ideally the fixes here should be very minimal and targeted.
>>>>>> >>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> So, IMHO, the key part is not release, it's branch.
>>>>>> >>>
>>>>>> >>> The first thing to decide is the branch.
>>>>>> >>>
>>>>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x
>>>>>> LTS
>>>>>> >>> branch. It's a branch where we will cherry-pick some important
>>>>>> fixes in
>>>>>> >>> the future and where we will cut release. It's the approach I use
>>>>>> in
>>>>>> >>> other Apache projects (especially Karaf) and it works fine.
>>>>>> >>
>>>>>> >>
>>>>>> >> JB, does Karaf has a documented process that we can re-use? If not
>>>>>> could you explain a bit more?
>>>>>> >>
>>>>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>>>>>> release out of that?
>>>>>> >>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> Just my $0.01
>>>>>> >>>
>>>>>> >>> Regards
>>>>>> >>> JB
>>>>>> >>>
>>>>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>>>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>>>>>> chamikara@google.com
>>>>>> >>> > <ma...@google.com>> wrote:
>>>>>> >>> >
>>>>>> >>> >
>>>>>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <
>>>>>> altay@google.com
>>>>>> >>> >     <ma...@google.com>> wrote:
>>>>>> >>> >
>>>>>> >>> >         I agree that LTS releases require more thought. Thank
>>>>>> you for
>>>>>> >>> >         raising these questions. What other open questions do
>>>>>> we have
>>>>>> >>> >         related LTS releases?
>>>>>> >>> >
>>>>>> >>> >         One way to do this would be to add them to a particular
>>>>>> tracking
>>>>>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be
>>>>>> ready for
>>>>>> >>> >         an LTS release ahead of time.
>>>>>> >>> >
>>>>>> >>> >         Related to dependencies, I agree with Thomas. If we
>>>>>> block on
>>>>>> >>> >         waiting for dependencies, we may end up taking a long
>>>>>> time
>>>>>> >>> >         before making any LTS release. And the reality of Beam
>>>>>> releases
>>>>>> >>> >         right now is that there are no supported releases today
>>>>>> that and
>>>>>> >>> >         in the long term that might hurt user trust. In my
>>>>>> opinion, we
>>>>>> >>> >         need to fix that sooner rather than later.
>>>>>> >>> >
>>>>>> >>> >
>>>>>> >>> >     Agree on the idea of focussing on stability instead of
>>>>>> feature set
>>>>>> >>> >     when it comes to LTS releases. Based on the previous
>>>>>> discussion on
>>>>>> >>> >     this [1] looks like the intended audience is enterprise
>>>>>> customers
>>>>>> >>> >     that would like to maintain a stable environment and that
>>>>>> usually
>>>>>> >>> >     have a long testing cycle before deploying a new version.
>>>>>> Seems
>>>>>> >>> >     like, rushing in features  or dependency upgrades for an
>>>>>> LTS release
>>>>>> >>> >     will be counterintuitive for such use cases.
>>>>>> >>> >
>>>>>> >>> >     On the other hand, I think a part of Ismaël point was that
>>>>>> all major
>>>>>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc
>>>>>> ..)
>>>>>> >>> >     should be supportable for the duration of the time Beam LTS
>>>>>> release
>>>>>> >>> >     is supported (1 year based on [1]). If a major dependency is
>>>>>> >>> >     expected to become unsupported within an year, it makes
>>>>>> sense to
>>>>>> >>> >     upgrade that before the LTS release. I suggest we do this
>>>>>> at least
>>>>>> >>> >     one release before the planned LTS release (so 2.9.0 if we
>>>>>> want to
>>>>>> >>> >     make 2.10.0 a LTS release) so that we have all major
>>>>>> >>> >     dependency/feature updates battle tested before committing
>>>>>> to them.
>>>>>> >>> >
>>>>>> >>> >
>>>>>> >>> > Dependencies are a significant concern. What happens if one of
>>>>>> our
>>>>>> >>> > dependencies has no version they're going to support for a year
>>>>>> hence?
>>>>>> >>> > It could happen that there's never a time when all our
>>>>>> dependencies will
>>>>>> >>> > be supported for at least a year as of any given date. And
>>>>>> rushing to
>>>>>> >>> > get the "latest" version of a dependency in is often contrary
>>>>>> to the
>>>>>> >>> > goals of an LTS. (Better to have those depend on the
>>>>>> battle-hardened
>>>>>> >>> > versions.)
>>>>>> >>> >
>>>>>> >>> > The long-term goal is probably to avoid pinning specific
>>>>>> versions as
>>>>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>>>>>> (whichever the
>>>>>> >>> > user provided), and considered backporting support for 1.7 when
>>>>>> it came
>>>>>> >>> > out, we'd be in much better shape here. How feasible this is
>>>>>> depends on
>>>>>> >>> > how backwards-compatible the dependency is.
>>>>>> >>> >
>>>>>> >>> > On the other hand, forcing a user to upgrade a (user-visible,
>>>>>> which in
>>>>>> >>> > particular includes runners) dependency seems contrary to the
>>>>>> goals of
>>>>>> >>> > an LTS. Even if that dependency becomes unsupported.
>>>>>> >>> >
>>>>>> >>> >
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Jean-Baptiste Onofré
>>>>>> >>> jbonofre@apache.org
>>>>>> >>> http://blog.nanthrax.net
>>>>>> >>> Talend - http://www.talend.com
>>>>>> >>
>>>>>> >>
>>>>>>
>>>>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Robert Bradshaw <ro...@google.com>]

On Sat, Oct 20, 2018 at 12:24 AM Kenneth Knowles <ke...@apache.org> wrote:

> Pinging this. I think Beam should have a live LTS branch.
>
> I want to suggest a different approach: choose something already released
> to be LTS. This way it has had some usage and we have some confidence there
> are no critical problems.
>

+1. This is what I suggested as well.


> So how about we make 2.7 the first LTS branch?
>

I would be fine with that, however, I think our first stab at an LTS branch
should not promise a full year of support, more like a test run. +1 to
Thomas' point that concrete user input/feedback would be useful in planning
here.


> Kenn
>
> On Wed, Oct 10, 2018 at 8:18 AM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> some times Ago JB spoke about Beam roadmap. I tend to think this
>> discussion does no make any sense without a clear roadmap. The rational
>> here is that a roadmap will give you the future changes
>> and the potential future versions (we spoke a few times of Beam 3). This
>> does not have to be very factual, a slice of 3 months is ok at that stage.
>> However, if you don't have that,
>> you can say 2.8 will be LTS and we support 2 versions but if 2.9 and 2.10
>> introduce breaking changes, then
>> it leads to a LTS 2.8 no more supported. This is just an example but the
>> ratio "cost(project) / gain(user)" 100% depends the plans for the project,
>> technically there is no blocker to support all releases
>> for life but would any PMC have the will to release beam 0.x now? The
>> point of a LTS for an user is to plan investment, if we are in previous
>> case it does not help IMHO.
>> So maybe grab back the Beam enhancement plans and assign them some fix
>> versions before defining what support model of Beam can be.
>>
>> Just the 2 cts of an outsider.
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>>
>> Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <ch...@google.com>
>> a écrit :
>>
>>>
>>>
>>> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> The simplest thing we can do is just to pin all the deps of the LTS
>>>>> and not move them in any maintenance release if not a strong reason to
>>>>> do so.
>>>>>
>>>>> The next subject is to make maintainers aware of which release will be
>>>>> the LTS in advance so they decide what to do with the dependencies
>>>>> versions. In my previous mail I mentioned all the possible cases that
>>>>> can happen with dependencies and it is clear that one unified policy
>>>>> won’t satisfy every one. So better let the maintainers (who can also
>>>>> ask for user feedback in the ML) to decide about  versions before the
>>>>> release.
>>>>>
>>>>> Alexey’s question is still a really important issue, and has been so
>>>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>>>>> Options are:
>>>>>
>>>>> (1) We keep consistent with Experimental which means that they are
>>>>> still not guarantees (note that this does not mean that they will be
>>>>> broken arbitrarily).
>>>>> (2) We are consistent with the LTS approach which makes them ‘non
>>>>> experimental’ for the LTS so we will guarantee the functionality/API
>>>>> stable.
>>>>>
>>>>> I personally have conflicted opinions I would like to favor (1) but
>>>>> this is not consistent with the whole idea of LTS so probably (2) is
>>>>> wiser.
>>>>>
>>>>
>>>> Yeah, I think (2) is the only viable option.
>>>>
>>>
>>> I think important thing here is that future releases on a  LTS branch
>>> will be patch (bugfix) releases so I don't think we can/should do
>>> API/functionality changes (even if the change is experimental and/or
>>> backwards compatible).
>>>
>>> I think same goes for dependency changes. If the change is to fix a
>>> known bug we can do that in a patch release but if it's to add more
>>> functionality probably that should come in a new minor release instead of a
>>> patch release.
>>>
>>> This is why I think we should be bit careful about "rushed" changes to
>>> major functionalities of Beam going into LTS releases. Just my 2 cents.
>>>
>>> Thanks,
>>> Cham
>>>
>>>
>>>>
>>>>
>>>>> Finally I also worry about Tim’s remarks on performance and quality,
>>>>> even if some of these things effectively can be fixed in a subsequent
>>>>> LTS release. Users will probably prefer a LTS to start with Beam and
>>>>> if the performance/quality of the LTS, this can hurt perception of the
>>>>> project.
>>>>>
>>>>
>>>> Yes, for this reason I think it's important to consider what goes into
>>>> an LTS as well as what happens after. Almost by definition, using an LTS is
>>>> choosing stability over cutting edge features. I'd rather major feature X
>>>> goes in after LTS, and lives in a couple of releases gaining fixes and
>>>> improvements before being released as part of the next LTS, than quickly
>>>> making it into an LTS while brand new (both due to the time period before
>>>> we refine it, and the extra work of porting refinements back).
>>>>
>>>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway,
>>>> waiting for at least x.y.1?
>>>>
>>>> Come to think of it, do we even have to designate releases at LTS at
>>>> the time of release? Perhaps we could instead just keep with our normal
>>>> release cadence, and periodically choose one of them as LTS once we've
>>>> confirmed its stability out in the wild and start backporting to it. (I can
>>>> think of several cons with this approach as well, e.g. generally it's
>>>> easier to backport bugfixes at the time a bugfix is done in master rather
>>>> than swapping in context at a later date, but just thought I'd throw it out
>>>> there.)
>>>>
>>>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > I've seen two mentions that "rushing" is contrary to the goals of
>>>>> LTS. But I wouldn't worry about this. The fact is there is almost nothing
>>>>> you can do to stabilize *prior* to cutting the LTS branch. Stability comes
>>>>> from the branch being long-lived and having multiple releases.
>>>>> >
>>>>> > (I think this is pretty much my version of what JB is saying)
>>>>> >
>>>>> > What a conservative user will do if 2.8.x is declared LTS is to
>>>>> start using the 2.8.x branch after it has had a couple bugfix releases. I
>>>>> don't think it is useful or possible to try for an "extra stable" 2.x.0.
>>>>> >
>>>>> > The arguments about supporting the most widely used versions of
>>>>> runner backends apply regardless of LTS. We should support them if we have
>>>>> the resources to do so.
>>>>> >
>>>>> > Kenn
>>>>> >
>>>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net> wrote:
>>>>> >>>
>>>>> >>> Hi,
>>>>> >>>
>>>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a
>>>>> branch
>>>>> >>> that we guarantee to have fixes on it for a long period of time.
>>>>> >>>
>>>>> >>>
>>>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>>>>> >>> releases on a LTS branch, the only constraint is to avoid to
>>>>> introduce
>>>>> >>> breaking changes.
>>>>> >>
>>>>> >>
>>>>> >> I agree with this perspective. Thank you for sharing this. However
>>>>> the other commenters also had a good point. Requiring users to upgrade
>>>>> their runner version maybe incompatible with the goals of an LTS branch.
>>>>> Ideally the fixes here should be very minimal and targeted.
>>>>> >>
>>>>> >>>
>>>>> >>>
>>>>> >>> So, IMHO, the key part is not release, it's branch.
>>>>> >>>
>>>>> >>> The first thing to decide is the branch.
>>>>> >>>
>>>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x
>>>>> LTS
>>>>> >>> branch. It's a branch where we will cherry-pick some important
>>>>> fixes in
>>>>> >>> the future and where we will cut release. It's the approach I use
>>>>> in
>>>>> >>> other Apache projects (especially Karaf) and it works fine.
>>>>> >>
>>>>> >>
>>>>> >> JB, does Karaf has a documented process that we can re-use? If not
>>>>> could you explain a bit more?
>>>>> >>
>>>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>>>>> release out of that?
>>>>> >>
>>>>> >>>
>>>>> >>>
>>>>> >>> Just my $0.01
>>>>> >>>
>>>>> >>> Regards
>>>>> >>> JB
>>>>> >>>
>>>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>>>>> chamikara@google.com
>>>>> >>> > <ma...@google.com>> wrote:
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>>>>> >>> >     <ma...@google.com>> wrote:
>>>>> >>> >
>>>>> >>> >         I agree that LTS releases require more thought. Thank
>>>>> you for
>>>>> >>> >         raising these questions. What other open questions do we
>>>>> have
>>>>> >>> >         related LTS releases?
>>>>> >>> >
>>>>> >>> >         One way to do this would be to add them to a particular
>>>>> tracking
>>>>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be
>>>>> ready for
>>>>> >>> >         an LTS release ahead of time.
>>>>> >>> >
>>>>> >>> >         Related to dependencies, I agree with Thomas. If we
>>>>> block on
>>>>> >>> >         waiting for dependencies, we may end up taking a long
>>>>> time
>>>>> >>> >         before making any LTS release. And the reality of Beam
>>>>> releases
>>>>> >>> >         right now is that there are no supported releases today
>>>>> that and
>>>>> >>> >         in the long term that might hurt user trust. In my
>>>>> opinion, we
>>>>> >>> >         need to fix that sooner rather than later.
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >     Agree on the idea of focussing on stability instead of
>>>>> feature set
>>>>> >>> >     when it comes to LTS releases. Based on the previous
>>>>> discussion on
>>>>> >>> >     this [1] looks like the intended audience is enterprise
>>>>> customers
>>>>> >>> >     that would like to maintain a stable environment and that
>>>>> usually
>>>>> >>> >     have a long testing cycle before deploying a new version.
>>>>> Seems
>>>>> >>> >     like, rushing in features  or dependency upgrades for an LTS
>>>>> release
>>>>> >>> >     will be counterintuitive for such use cases.
>>>>> >>> >
>>>>> >>> >     On the other hand, I think a part of Ismaël point was that
>>>>> all major
>>>>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc
>>>>> ..)
>>>>> >>> >     should be supportable for the duration of the time Beam LTS
>>>>> release
>>>>> >>> >     is supported (1 year based on [1]). If a major dependency is
>>>>> >>> >     expected to become unsupported within an year, it makes
>>>>> sense to
>>>>> >>> >     upgrade that before the LTS release. I suggest we do this at
>>>>> least
>>>>> >>> >     one release before the planned LTS release (so 2.9.0 if we
>>>>> want to
>>>>> >>> >     make 2.10.0 a LTS release) so that we have all major
>>>>> >>> >     dependency/feature updates battle tested before committing
>>>>> to them.
>>>>> >>> >
>>>>> >>> >
>>>>> >>> > Dependencies are a significant concern. What happens if one of
>>>>> our
>>>>> >>> > dependencies has no version they're going to support for a year
>>>>> hence?
>>>>> >>> > It could happen that there's never a time when all our
>>>>> dependencies will
>>>>> >>> > be supported for at least a year as of any given date. And
>>>>> rushing to
>>>>> >>> > get the "latest" version of a dependency in is often contrary to
>>>>> the
>>>>> >>> > goals of an LTS. (Better to have those depend on the
>>>>> battle-hardened
>>>>> >>> > versions.)
>>>>> >>> >
>>>>> >>> > The long-term goal is probably to avoid pinning specific
>>>>> versions as
>>>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>>>>> (whichever the
>>>>> >>> > user provided), and considered backporting support for 1.7 when
>>>>> it came
>>>>> >>> > out, we'd be in much better shape here. How feasible this is
>>>>> depends on
>>>>> >>> > how backwards-compatible the dependency is.
>>>>> >>> >
>>>>> >>> > On the other hand, forcing a user to upgrade a (user-visible,
>>>>> which in
>>>>> >>> > particular includes runners) dependency seems contrary to the
>>>>> goals of
>>>>> >>> > an LTS. Even if that dependency becomes unsupported.
>>>>> >>> >
>>>>> >>> >
>>>>> >>>
>>>>> >>> --
>>>>> >>> Jean-Baptiste Onofré
>>>>> >>> jbonofre@apache.org
>>>>> >>> http://blog.nanthrax.net
>>>>> >>> Talend - http://www.talend.com
>>>>> >>
>>>>> >>
>>>>>
>>>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Kenneth Knowles <ke...@apache.org>]

Pinging this. I think Beam should have a live LTS branch.

I want to suggest a different approach: choose something already released
to be LTS. This way it has had some usage and we have some confidence there
are no critical problems.

So how about we make 2.7 the first LTS branch?

Kenn

On Wed, Oct 10, 2018 at 8:18 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:

> some times Ago JB spoke about Beam roadmap. I tend to think this
> discussion does no make any sense without a clear roadmap. The rational
> here is that a roadmap will give you the future changes
> and the potential future versions (we spoke a few times of Beam 3). This
> does not have to be very factual, a slice of 3 months is ok at that stage.
> However, if you don't have that,
> you can say 2.8 will be LTS and we support 2 versions but if 2.9 and 2.10
> introduce breaking changes, then
> it leads to a LTS 2.8 no more supported. This is just an example but the
> ratio "cost(project) / gain(user)" 100% depends the plans for the project,
> technically there is no blocker to support all releases
> for life but would any PMC have the will to release beam 0.x now? The
> point of a LTS for an user is to plan investment, if we are in previous
> case it does not help IMHO.
> So maybe grab back the Beam enhancement plans and assign them some fix
> versions before defining what support model of Beam can be.
>
> Just the 2 cts of an outsider.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <ch...@google.com> a
> écrit :
>
>>
>>
>> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> The simplest thing we can do is just to pin all the deps of the LTS
>>>> and not move them in any maintenance release if not a strong reason to
>>>> do so.
>>>>
>>>> The next subject is to make maintainers aware of which release will be
>>>> the LTS in advance so they decide what to do with the dependencies
>>>> versions. In my previous mail I mentioned all the possible cases that
>>>> can happen with dependencies and it is clear that one unified policy
>>>> won’t satisfy every one. So better let the maintainers (who can also
>>>> ask for user feedback in the ML) to decide about  versions before the
>>>> release.
>>>>
>>>> Alexey’s question is still a really important issue, and has been so
>>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>>>> Options are:
>>>>
>>>> (1) We keep consistent with Experimental which means that they are
>>>> still not guarantees (note that this does not mean that they will be
>>>> broken arbitrarily).
>>>> (2) We are consistent with the LTS approach which makes them ‘non
>>>> experimental’ for the LTS so we will guarantee the functionality/API
>>>> stable.
>>>>
>>>> I personally have conflicted opinions I would like to favor (1) but
>>>> this is not consistent with the whole idea of LTS so probably (2) is
>>>> wiser.
>>>>
>>>
>>> Yeah, I think (2) is the only viable option.
>>>
>>
>> I think important thing here is that future releases on a  LTS branch
>> will be patch (bugfix) releases so I don't think we can/should do
>> API/functionality changes (even if the change is experimental and/or
>> backwards compatible).
>>
>> I think same goes for dependency changes. If the change is to fix a known
>> bug we can do that in a patch release but if it's to add more functionality
>> probably that should come in a new minor release instead of a patch
>> release.
>>
>> This is why I think we should be bit careful about "rushed" changes to
>> major functionalities of Beam going into LTS releases. Just my 2 cents.
>>
>> Thanks,
>> Cham
>>
>>
>>>
>>>
>>>> Finally I also worry about Tim’s remarks on performance and quality,
>>>> even if some of these things effectively can be fixed in a subsequent
>>>> LTS release. Users will probably prefer a LTS to start with Beam and
>>>> if the performance/quality of the LTS, this can hurt perception of the
>>>> project.
>>>>
>>>
>>> Yes, for this reason I think it's important to consider what goes into
>>> an LTS as well as what happens after. Almost by definition, using an LTS is
>>> choosing stability over cutting edge features. I'd rather major feature X
>>> goes in after LTS, and lives in a couple of releases gaining fixes and
>>> improvements before being released as part of the next LTS, than quickly
>>> making it into an LTS while brand new (both due to the time period before
>>> we refine it, and the extra work of porting refinements back).
>>>
>>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway,
>>> waiting for at least x.y.1?
>>>
>>> Come to think of it, do we even have to designate releases at LTS at the
>>> time of release? Perhaps we could instead just keep with our normal release
>>> cadence, and periodically choose one of them as LTS once we've confirmed
>>> its stability out in the wild and start backporting to it. (I can think of
>>> several cons with this approach as well, e.g. generally it's easier to
>>> backport bugfixes at the time a bugfix is done in master rather than
>>> swapping in context at a later date, but just thought I'd throw it out
>>> there.)
>>>
>>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org> wrote:
>>>> >
>>>> > I've seen two mentions that "rushing" is contrary to the goals of
>>>> LTS. But I wouldn't worry about this. The fact is there is almost nothing
>>>> you can do to stabilize *prior* to cutting the LTS branch. Stability comes
>>>> from the branch being long-lived and having multiple releases.
>>>> >
>>>> > (I think this is pretty much my version of what JB is saying)
>>>> >
>>>> > What a conservative user will do if 2.8.x is declared LTS is to start
>>>> using the 2.8.x branch after it has had a couple bugfix releases. I don't
>>>> think it is useful or possible to try for an "extra stable" 2.x.0.
>>>> >
>>>> > The arguments about supporting the most widely used versions of
>>>> runner backends apply regardless of LTS. We should support them if we have
>>>> the resources to do so.
>>>> >
>>>> > Kenn
>>>> >
>>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net> wrote:
>>>> >>>
>>>> >>> Hi,
>>>> >>>
>>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a
>>>> branch
>>>> >>> that we guarantee to have fixes on it for a long period of time.
>>>> >>>
>>>> >>>
>>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>>>> >>> releases on a LTS branch, the only constraint is to avoid to
>>>> introduce
>>>> >>> breaking changes.
>>>> >>
>>>> >>
>>>> >> I agree with this perspective. Thank you for sharing this. However
>>>> the other commenters also had a good point. Requiring users to upgrade
>>>> their runner version maybe incompatible with the goals of an LTS branch.
>>>> Ideally the fixes here should be very minimal and targeted.
>>>> >>
>>>> >>>
>>>> >>>
>>>> >>> So, IMHO, the key part is not release, it's branch.
>>>> >>>
>>>> >>> The first thing to decide is the branch.
>>>> >>>
>>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
>>>> >>> branch. It's a branch where we will cherry-pick some important
>>>> fixes in
>>>> >>> the future and where we will cut release. It's the approach I use in
>>>> >>> other Apache projects (especially Karaf) and it works fine.
>>>> >>
>>>> >>
>>>> >> JB, does Karaf has a documented process that we can re-use? If not
>>>> could you explain a bit more?
>>>> >>
>>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>>>> release out of that?
>>>> >>
>>>> >>>
>>>> >>>
>>>> >>> Just my $0.01
>>>> >>>
>>>> >>> Regards
>>>> >>> JB
>>>> >>>
>>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>>>> chamikara@google.com
>>>> >>> > <ma...@google.com>> wrote:
>>>> >>> >
>>>> >>> >
>>>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>>>> >>> >     <ma...@google.com>> wrote:
>>>> >>> >
>>>> >>> >         I agree that LTS releases require more thought. Thank you
>>>> for
>>>> >>> >         raising these questions. What other open questions do we
>>>> have
>>>> >>> >         related LTS releases?
>>>> >>> >
>>>> >>> >         One way to do this would be to add them to a particular
>>>> tracking
>>>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be
>>>> ready for
>>>> >>> >         an LTS release ahead of time.
>>>> >>> >
>>>> >>> >         Related to dependencies, I agree with Thomas. If we block
>>>> on
>>>> >>> >         waiting for dependencies, we may end up taking a long time
>>>> >>> >         before making any LTS release. And the reality of Beam
>>>> releases
>>>> >>> >         right now is that there are no supported releases today
>>>> that and
>>>> >>> >         in the long term that might hurt user trust. In my
>>>> opinion, we
>>>> >>> >         need to fix that sooner rather than later.
>>>> >>> >
>>>> >>> >
>>>> >>> >     Agree on the idea of focussing on stability instead of
>>>> feature set
>>>> >>> >     when it comes to LTS releases. Based on the previous
>>>> discussion on
>>>> >>> >     this [1] looks like the intended audience is enterprise
>>>> customers
>>>> >>> >     that would like to maintain a stable environment and that
>>>> usually
>>>> >>> >     have a long testing cycle before deploying a new version.
>>>> Seems
>>>> >>> >     like, rushing in features  or dependency upgrades for an LTS
>>>> release
>>>> >>> >     will be counterintuitive for such use cases.
>>>> >>> >
>>>> >>> >     On the other hand, I think a part of Ismaël point was that
>>>> all major
>>>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc
>>>> ..)
>>>> >>> >     should be supportable for the duration of the time Beam LTS
>>>> release
>>>> >>> >     is supported (1 year based on [1]). If a major dependency is
>>>> >>> >     expected to become unsupported within an year, it makes sense
>>>> to
>>>> >>> >     upgrade that before the LTS release. I suggest we do this at
>>>> least
>>>> >>> >     one release before the planned LTS release (so 2.9.0 if we
>>>> want to
>>>> >>> >     make 2.10.0 a LTS release) so that we have all major
>>>> >>> >     dependency/feature updates battle tested before committing to
>>>> them.
>>>> >>> >
>>>> >>> >
>>>> >>> > Dependencies are a significant concern. What happens if one of our
>>>> >>> > dependencies has no version they're going to support for a year
>>>> hence?
>>>> >>> > It could happen that there's never a time when all our
>>>> dependencies will
>>>> >>> > be supported for at least a year as of any given date. And
>>>> rushing to
>>>> >>> > get the "latest" version of a dependency in is often contrary to
>>>> the
>>>> >>> > goals of an LTS. (Better to have those depend on the
>>>> battle-hardened
>>>> >>> > versions.)
>>>> >>> >
>>>> >>> > The long-term goal is probably to avoid pinning specific versions
>>>> as
>>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>>>> (whichever the
>>>> >>> > user provided), and considered backporting support for 1.7 when
>>>> it came
>>>> >>> > out, we'd be in much better shape here. How feasible this is
>>>> depends on
>>>> >>> > how backwards-compatible the dependency is.
>>>> >>> >
>>>> >>> > On the other hand, forcing a user to upgrade a (user-visible,
>>>> which in
>>>> >>> > particular includes runners) dependency seems contrary to the
>>>> goals of
>>>> >>> > an LTS. Even if that dependency becomes unsupported.
>>>> >>> >
>>>> >>> >
>>>> >>>
>>>> >>> --
>>>> >>> Jean-Baptiste Onofré
>>>> >>> jbonofre@apache.org
>>>> >>> http://blog.nanthrax.net
>>>> >>> Talend - http://www.talend.com
>>>> >>
>>>> >>
>>>>
>>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Romain Manni-Bucau <rm...@gmail.com>]

some times Ago JB spoke about Beam roadmap. I tend to think this discussion
does no make any sense without a clear roadmap. The rational here is that a
roadmap will give you the future changes
and the potential future versions (we spoke a few times of Beam 3). This
does not have to be very factual, a slice of 3 months is ok at that stage.
However, if you don't have that,
you can say 2.8 will be LTS and we support 2 versions but if 2.9 and 2.10
introduce breaking changes, then
it leads to a LTS 2.8 no more supported. This is just an example but the
ratio "cost(project) / gain(user)" 100% depends the plans for the project,
technically there is no blocker to support all releases
for life but would any PMC have the will to release beam 0.x now? The point
of a LTS for an user is to plan investment, if we are in previous case it
does not help IMHO.
So maybe grab back the Beam enhancement plans and assign them some fix
versions before defining what support model of Beam can be.

Just the 2 cts of an outsider.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 10 oct. 2018 à 17:10, Chamikara Jayalath <ch...@google.com> a
écrit :

>
>
> On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> The simplest thing we can do is just to pin all the deps of the LTS
>>> and not move them in any maintenance release if not a strong reason to
>>> do so.
>>>
>>> The next subject is to make maintainers aware of which release will be
>>> the LTS in advance so they decide what to do with the dependencies
>>> versions. In my previous mail I mentioned all the possible cases that
>>> can happen with dependencies and it is clear that one unified policy
>>> won’t satisfy every one. So better let the maintainers (who can also
>>> ask for user feedback in the ML) to decide about  versions before the
>>> release.
>>>
>>> Alexey’s question is still a really important issue, and has been so
>>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>>> Options are:
>>>
>>> (1) We keep consistent with Experimental which means that they are
>>> still not guarantees (note that this does not mean that they will be
>>> broken arbitrarily).
>>> (2) We are consistent with the LTS approach which makes them ‘non
>>> experimental’ for the LTS so we will guarantee the functionality/API
>>> stable.
>>>
>>> I personally have conflicted opinions I would like to favor (1) but
>>> this is not consistent with the whole idea of LTS so probably (2) is
>>> wiser.
>>>
>>
>> Yeah, I think (2) is the only viable option.
>>
>
> I think important thing here is that future releases on a  LTS branch will
> be patch (bugfix) releases so I don't think we can/should do
> API/functionality changes (even if the change is experimental and/or
> backwards compatible).
>
> I think same goes for dependency changes. If the change is to fix a known
> bug we can do that in a patch release but if it's to add more functionality
> probably that should come in a new minor release instead of a patch
> release.
>
> This is why I think we should be bit careful about "rushed" changes to
> major functionalities of Beam going into LTS releases. Just my 2 cents.
>
> Thanks,
> Cham
>
>
>>
>>
>>> Finally I also worry about Tim’s remarks on performance and quality,
>>> even if some of these things effectively can be fixed in a subsequent
>>> LTS release. Users will probably prefer a LTS to start with Beam and
>>> if the performance/quality of the LTS, this can hurt perception of the
>>> project.
>>>
>>
>> Yes, for this reason I think it's important to consider what goes into an
>> LTS as well as what happens after. Almost by definition, using an LTS is
>> choosing stability over cutting edge features. I'd rather major feature X
>> goes in after LTS, and lives in a couple of releases gaining fixes and
>> improvements before being released as part of the next LTS, than quickly
>> making it into an LTS while brand new (both due to the time period before
>> we refine it, and the extra work of porting refinements back).
>>
>> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway,
>> waiting for at least x.y.1?
>>
>> Come to think of it, do we even have to designate releases at LTS at the
>> time of release? Perhaps we could instead just keep with our normal release
>> cadence, and periodically choose one of them as LTS once we've confirmed
>> its stability out in the wild and start backporting to it. (I can think of
>> several cons with this approach as well, e.g. generally it's easier to
>> backport bugfixes at the time a bugfix is done in master rather than
>> swapping in context at a later date, but just thought I'd throw it out
>> there.)
>>
>> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org> wrote:
>>> >
>>> > I've seen two mentions that "rushing" is contrary to the goals of LTS.
>>> But I wouldn't worry about this. The fact is there is almost nothing you
>>> can do to stabilize *prior* to cutting the LTS branch. Stability comes from
>>> the branch being long-lived and having multiple releases.
>>> >
>>> > (I think this is pretty much my version of what JB is saying)
>>> >
>>> > What a conservative user will do if 2.8.x is declared LTS is to start
>>> using the 2.8.x branch after it has had a couple bugfix releases. I don't
>>> think it is useful or possible to try for an "extra stable" 2.x.0.
>>> >
>>> > The arguments about supporting the most widely used versions of runner
>>> backends apply regardless of LTS. We should support them if we have the
>>> resources to do so.
>>> >
>>> > Kenn
>>> >
>>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> I think we have to remember what it's a LTS. A LTS is clearly a
>>> branch
>>> >>> that we guarantee to have fixes on it for a long period of time.
>>> >>>
>>> >>>
>>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>>> >>> releases on a LTS branch, the only constraint is to avoid to
>>> introduce
>>> >>> breaking changes.
>>> >>
>>> >>
>>> >> I agree with this perspective. Thank you for sharing this. However
>>> the other commenters also had a good point. Requiring users to upgrade
>>> their runner version maybe incompatible with the goals of an LTS branch.
>>> Ideally the fixes here should be very minimal and targeted.
>>> >>
>>> >>>
>>> >>>
>>> >>> So, IMHO, the key part is not release, it's branch.
>>> >>>
>>> >>> The first thing to decide is the branch.
>>> >>>
>>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
>>> >>> branch. It's a branch where we will cherry-pick some important fixes
>>> in
>>> >>> the future and where we will cut release. It's the approach I use in
>>> >>> other Apache projects (especially Karaf) and it works fine.
>>> >>
>>> >>
>>> >> JB, does Karaf has a documented process that we can re-use? If not
>>> could you explain a bit more?
>>> >>
>>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>>> release out of that?
>>> >>
>>> >>>
>>> >>>
>>> >>> Just my $0.01
>>> >>>
>>> >>> Regards
>>> >>> JB
>>> >>>
>>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>>> chamikara@google.com
>>> >>> > <ma...@google.com>> wrote:
>>> >>> >
>>> >>> >
>>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>>> >>> >     <ma...@google.com>> wrote:
>>> >>> >
>>> >>> >         I agree that LTS releases require more thought. Thank you
>>> for
>>> >>> >         raising these questions. What other open questions do we
>>> have
>>> >>> >         related LTS releases?
>>> >>> >
>>> >>> >         One way to do this would be to add them to a particular
>>> tracking
>>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be ready
>>> for
>>> >>> >         an LTS release ahead of time.
>>> >>> >
>>> >>> >         Related to dependencies, I agree with Thomas. If we block
>>> on
>>> >>> >         waiting for dependencies, we may end up taking a long time
>>> >>> >         before making any LTS release. And the reality of Beam
>>> releases
>>> >>> >         right now is that there are no supported releases today
>>> that and
>>> >>> >         in the long term that might hurt user trust. In my
>>> opinion, we
>>> >>> >         need to fix that sooner rather than later.
>>> >>> >
>>> >>> >
>>> >>> >     Agree on the idea of focussing on stability instead of feature
>>> set
>>> >>> >     when it comes to LTS releases. Based on the previous
>>> discussion on
>>> >>> >     this [1] looks like the intended audience is enterprise
>>> customers
>>> >>> >     that would like to maintain a stable environment and that
>>> usually
>>> >>> >     have a long testing cycle before deploying a new version. Seems
>>> >>> >     like, rushing in features  or dependency upgrades for an LTS
>>> release
>>> >>> >     will be counterintuitive for such use cases.
>>> >>> >
>>> >>> >     On the other hand, I think a part of Ismaël point was that all
>>> major
>>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
>>> >>> >     should be supportable for the duration of the time Beam LTS
>>> release
>>> >>> >     is supported (1 year based on [1]). If a major dependency is
>>> >>> >     expected to become unsupported within an year, it makes sense
>>> to
>>> >>> >     upgrade that before the LTS release. I suggest we do this at
>>> least
>>> >>> >     one release before the planned LTS release (so 2.9.0 if we
>>> want to
>>> >>> >     make 2.10.0 a LTS release) so that we have all major
>>> >>> >     dependency/feature updates battle tested before committing to
>>> them.
>>> >>> >
>>> >>> >
>>> >>> > Dependencies are a significant concern. What happens if one of our
>>> >>> > dependencies has no version they're going to support for a year
>>> hence?
>>> >>> > It could happen that there's never a time when all our
>>> dependencies will
>>> >>> > be supported for at least a year as of any given date. And rushing
>>> to
>>> >>> > get the "latest" version of a dependency in is often contrary to
>>> the
>>> >>> > goals of an LTS. (Better to have those depend on the
>>> battle-hardened
>>> >>> > versions.)
>>> >>> >
>>> >>> > The long-term goal is probably to avoid pinning specific versions
>>> as
>>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>>> (whichever the
>>> >>> > user provided), and considered backporting support for 1.7 when it
>>> came
>>> >>> > out, we'd be in much better shape here. How feasible this is
>>> depends on
>>> >>> > how backwards-compatible the dependency is.
>>> >>> >
>>> >>> > On the other hand, forcing a user to upgrade a (user-visible,
>>> which in
>>> >>> > particular includes runners) dependency seems contrary to the
>>> goals of
>>> >>> > an LTS. Even if that dependency becomes unsupported.
>>> >>> >
>>> >>> >
>>> >>>
>>> >>> --
>>> >>> Jean-Baptiste Onofré
>>> >>> jbonofre@apache.org
>>> >>> http://blog.nanthrax.net
>>> >>> Talend - http://www.talend.com
>>> >>
>>> >>
>>>
>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Chamikara Jayalath <ch...@google.com>]

On Wed, Oct 10, 2018 at 2:56 AM Robert Bradshaw <ro...@google.com> wrote:

> On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> The simplest thing we can do is just to pin all the deps of the LTS
>> and not move them in any maintenance release if not a strong reason to
>> do so.
>>
>> The next subject is to make maintainers aware of which release will be
>> the LTS in advance so they decide what to do with the dependencies
>> versions. In my previous mail I mentioned all the possible cases that
>> can happen with dependencies and it is clear that one unified policy
>> won’t satisfy every one. So better let the maintainers (who can also
>> ask for user feedback in the ML) to decide about  versions before the
>> release.
>>
>> Alexey’s question is still a really important issue, and has been so
>> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
>> Options are:
>>
>> (1) We keep consistent with Experimental which means that they are
>> still not guarantees (note that this does not mean that they will be
>> broken arbitrarily).
>> (2) We are consistent with the LTS approach which makes them ‘non
>> experimental’ for the LTS so we will guarantee the functionality/API
>> stable.
>>
>> I personally have conflicted opinions I would like to favor (1) but
>> this is not consistent with the whole idea of LTS so probably (2) is
>> wiser.
>>
>
> Yeah, I think (2) is the only viable option.
>

I think important thing here is that future releases on a  LTS branch will
be patch (bugfix) releases so I don't think we can/should do
API/functionality changes (even if the change is experimental and/or
backwards compatible).

I think same goes for dependency changes. If the change is to fix a known
bug we can do that in a patch release but if it's to add more functionality
probably that should come in a new minor release instead of a patch
release.

This is why I think we should be bit careful about "rushed" changes to
major functionalities of Beam going into LTS releases. Just my 2 cents.

Thanks,
Cham


>
>
>> Finally I also worry about Tim’s remarks on performance and quality,
>> even if some of these things effectively can be fixed in a subsequent
>> LTS release. Users will probably prefer a LTS to start with Beam and
>> if the performance/quality of the LTS, this can hurt perception of the
>> project.
>>
>
> Yes, for this reason I think it's important to consider what goes into an
> LTS as well as what happens after. Almost by definition, using an LTS is
> choosing stability over cutting edge features. I'd rather major feature X
> goes in after LTS, and lives in a couple of releases gaining fixes and
> improvements before being released as part of the next LTS, than quickly
> making it into an LTS while brand new (both due to the time period before
> we refine it, and the extra work of porting refinements back).
>
> Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway, waiting
> for at least x.y.1?
>
> Come to think of it, do we even have to designate releases at LTS at the
> time of release? Perhaps we could instead just keep with our normal release
> cadence, and periodically choose one of them as LTS once we've confirmed
> its stability out in the wild and start backporting to it. (I can think of
> several cons with this approach as well, e.g. generally it's easier to
> backport bugfixes at the time a bugfix is done in master rather than
> swapping in context at a later date, but just thought I'd throw it out
> there.)
>
> On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > I've seen two mentions that "rushing" is contrary to the goals of LTS.
>> But I wouldn't worry about this. The fact is there is almost nothing you
>> can do to stabilize *prior* to cutting the LTS branch. Stability comes from
>> the branch being long-lived and having multiple releases.
>> >
>> > (I think this is pretty much my version of what JB is saying)
>> >
>> > What a conservative user will do if 2.8.x is declared LTS is to start
>> using the 2.8.x branch after it has had a couple bugfix releases. I don't
>> think it is useful or possible to try for an "extra stable" 2.x.0.
>> >
>> > The arguments about supporting the most widely used versions of runner
>> backends apply regardless of LTS. We should support them if we have the
>> resources to do so.
>> >
>> > Kenn
>> >
>> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>> >>
>> >>
>> >>
>> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> I think we have to remember what it's a LTS. A LTS is clearly a branch
>> >>> that we guarantee to have fixes on it for a long period of time.
>> >>>
>> >>>
>> >>> It doesn't mean that LTS == unique release. We can do a bunch of
>> >>> releases on a LTS branch, the only constraint is to avoid to introduce
>> >>> breaking changes.
>> >>
>> >>
>> >> I agree with this perspective. Thank you for sharing this. However the
>> other commenters also had a good point. Requiring users to upgrade their
>> runner version maybe incompatible with the goals of an LTS branch. Ideally
>> the fixes here should be very minimal and targeted.
>> >>
>> >>>
>> >>>
>> >>> So, IMHO, the key part is not release, it's branch.
>> >>>
>> >>> The first thing to decide is the branch.
>> >>>
>> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
>> >>> branch. It's a branch where we will cherry-pick some important fixes
>> in
>> >>> the future and where we will cut release. It's the approach I use in
>> >>> other Apache projects (especially Karaf) and it works fine.
>> >>
>> >>
>> >> JB, does Karaf has a documented process that we can re-use? If not
>> could you explain a bit more?
>> >>
>> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
>> release out of that?
>> >>
>> >>>
>> >>>
>> >>> Just my $0.01
>> >>>
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
>> chamikara@google.com
>> >>> > <ma...@google.com>> wrote:
>> >>> >
>> >>> >
>> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>> >>> >     <ma...@google.com>> wrote:
>> >>> >
>> >>> >         I agree that LTS releases require more thought. Thank you
>> for
>> >>> >         raising these questions. What other open questions do we
>> have
>> >>> >         related LTS releases?
>> >>> >
>> >>> >         One way to do this would be to add them to a particular
>> tracking
>> >>> >         list (e.g. 2.9.0 blocking list) that way we would be ready
>> for
>> >>> >         an LTS release ahead of time.
>> >>> >
>> >>> >         Related to dependencies, I agree with Thomas. If we block on
>> >>> >         waiting for dependencies, we may end up taking a long time
>> >>> >         before making any LTS release. And the reality of Beam
>> releases
>> >>> >         right now is that there are no supported releases today
>> that and
>> >>> >         in the long term that might hurt user trust. In my opinion,
>> we
>> >>> >         need to fix that sooner rather than later.
>> >>> >
>> >>> >
>> >>> >     Agree on the idea of focussing on stability instead of feature
>> set
>> >>> >     when it comes to LTS releases. Based on the previous discussion
>> on
>> >>> >     this [1] looks like the intended audience is enterprise
>> customers
>> >>> >     that would like to maintain a stable environment and that
>> usually
>> >>> >     have a long testing cycle before deploying a new version. Seems
>> >>> >     like, rushing in features  or dependency upgrades for an LTS
>> release
>> >>> >     will be counterintuitive for such use cases.
>> >>> >
>> >>> >     On the other hand, I think a part of Ismaël point was that all
>> major
>> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
>> >>> >     should be supportable for the duration of the time Beam LTS
>> release
>> >>> >     is supported (1 year based on [1]). If a major dependency is
>> >>> >     expected to become unsupported within an year, it makes sense to
>> >>> >     upgrade that before the LTS release. I suggest we do this at
>> least
>> >>> >     one release before the planned LTS release (so 2.9.0 if we want
>> to
>> >>> >     make 2.10.0 a LTS release) so that we have all major
>> >>> >     dependency/feature updates battle tested before committing to
>> them.
>> >>> >
>> >>> >
>> >>> > Dependencies are a significant concern. What happens if one of our
>> >>> > dependencies has no version they're going to support for a year
>> hence?
>> >>> > It could happen that there's never a time when all our dependencies
>> will
>> >>> > be supported for at least a year as of any given date. And rushing
>> to
>> >>> > get the "latest" version of a dependency in is often contrary to the
>> >>> > goals of an LTS. (Better to have those depend on the battle-hardened
>> >>> > versions.)
>> >>> >
>> >>> > The long-term goal is probably to avoid pinning specific versions as
>> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6
>> (whichever the
>> >>> > user provided), and considered backporting support for 1.7 when it
>> came
>> >>> > out, we'd be in much better shape here. How feasible this is
>> depends on
>> >>> > how backwards-compatible the dependency is.
>> >>> >
>> >>> > On the other hand, forcing a user to upgrade a (user-visible, which
>> in
>> >>> > particular includes runners) dependency seems contrary to the goals
>> of
>> >>> > an LTS. Even if that dependency becomes unsupported.
>> >>> >
>> >>> >
>> >>>
>> >>> --
>> >>> Jean-Baptiste Onofré
>> >>> jbonofre@apache.org
>> >>> http://blog.nanthrax.net
>> >>> Talend - http://www.talend.com
>> >>
>> >>
>>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Robert Bradshaw <ro...@google.com>]

On Wed, Oct 10, 2018 at 9:37 AM Ismaël Mejía <ie...@gmail.com> wrote:

> The simplest thing we can do is just to pin all the deps of the LTS
> and not move them in any maintenance release if not a strong reason to
> do so.
>
> The next subject is to make maintainers aware of which release will be
> the LTS in advance so they decide what to do with the dependencies
> versions. In my previous mail I mentioned all the possible cases that
> can happen with dependencies and it is clear that one unified policy
> won’t satisfy every one. So better let the maintainers (who can also
> ask for user feedback in the ML) to decide about  versions before the
> release.
>
> Alexey’s question is still a really important issue, and has been so
> far ignored. What happens with the ‘Experimental’ APIs in the LTS.
> Options are:
>
> (1) We keep consistent with Experimental which means that they are
> still not guarantees (note that this does not mean that they will be
> broken arbitrarily).
> (2) We are consistent with the LTS approach which makes them ‘non
> experimental’ for the LTS so we will guarantee the functionality/API
> stable.
>
> I personally have conflicted opinions I would like to favor (1) but
> this is not consistent with the whole idea of LTS so probably (2) is
> wiser.
>

Yeah, I think (2) is the only viable option.


> Finally I also worry about Tim’s remarks on performance and quality,
> even if some of these things effectively can be fixed in a subsequent
> LTS release. Users will probably prefer a LTS to start with Beam and
> if the performance/quality of the LTS, this can hurt perception of the
> project.
>

Yes, for this reason I think it's important to consider what goes into an
LTS as well as what happens after. Almost by definition, using an LTS is
choosing stability over cutting edge features. I'd rather major feature X
goes in after LTS, and lives in a couple of releases gaining fixes and
improvements before being released as part of the next LTS, than quickly
making it into an LTS while brand new (both due to the time period before
we refine it, and the extra work of porting refinements back).

Or maybe LTS-users are unlikely to pick up a x.y.0 release anyway, waiting
for at least x.y.1?

Come to think of it, do we even have to designate releases at LTS at the
time of release? Perhaps we could instead just keep with our normal release
cadence, and periodically choose one of them as LTS once we've confirmed
its stability out in the wild and start backporting to it. (I can think of
several cons with this approach as well, e.g. generally it's easier to
backport bugfixes at the time a bugfix is done in master rather than
swapping in context at a later date, but just thought I'd throw it out
there.)

On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org> wrote:
> >
> > I've seen two mentions that "rushing" is contrary to the goals of LTS.
> But I wouldn't worry about this. The fact is there is almost nothing you
> can do to stabilize *prior* to cutting the LTS branch. Stability comes from
> the branch being long-lived and having multiple releases.
> >
> > (I think this is pretty much my version of what JB is saying)
> >
> > What a conservative user will do if 2.8.x is declared LTS is to start
> using the 2.8.x branch after it has had a couple bugfix releases. I don't
> think it is useful or possible to try for an "extra stable" 2.x.0.
> >
> > The arguments about supporting the most widely used versions of runner
> backends apply regardless of LTS. We should support them if we have the
> resources to do so.
> >
> > Kenn
> >
> > On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
> >>
> >>
> >>
> >> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I think we have to remember what it's a LTS. A LTS is clearly a branch
> >>> that we guarantee to have fixes on it for a long period of time.
> >>>
> >>>
> >>> It doesn't mean that LTS == unique release. We can do a bunch of
> >>> releases on a LTS branch, the only constraint is to avoid to introduce
> >>> breaking changes.
> >>
> >>
> >> I agree with this perspective. Thank you for sharing this. However the
> other commenters also had a good point. Requiring users to upgrade their
> runner version maybe incompatible with the goals of an LTS branch. Ideally
> the fixes here should be very minimal and targeted.
> >>
> >>>
> >>>
> >>> So, IMHO, the key part is not release, it's branch.
> >>>
> >>> The first thing to decide is the branch.
> >>>
> >>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
> >>> branch. It's a branch where we will cherry-pick some important fixes in
> >>> the future and where we will cut release. It's the approach I use in
> >>> other Apache projects (especially Karaf) and it works fine.
> >>
> >>
> >> JB, does Karaf has a documented process that we can re-use? If not
> could you explain a bit more?
> >>
> >> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0
> release out of that?
> >>
> >>>
> >>>
> >>> Just my $0.01
> >>>
> >>> Regards
> >>> JB
> >>>
> >>> On 05/10/2018 12:14, Robert Bradshaw wrote:
> >>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <
> chamikara@google.com
> >>> > <ma...@google.com>> wrote:
> >>> >
> >>> >
> >>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
> >>> >     <ma...@google.com>> wrote:
> >>> >
> >>> >         I agree that LTS releases require more thought. Thank you for
> >>> >         raising these questions. What other open questions do we have
> >>> >         related LTS releases?
> >>> >
> >>> >         One way to do this would be to add them to a particular
> tracking
> >>> >         list (e.g. 2.9.0 blocking list) that way we would be ready
> for
> >>> >         an LTS release ahead of time.
> >>> >
> >>> >         Related to dependencies, I agree with Thomas. If we block on
> >>> >         waiting for dependencies, we may end up taking a long time
> >>> >         before making any LTS release. And the reality of Beam
> releases
> >>> >         right now is that there are no supported releases today that
> and
> >>> >         in the long term that might hurt user trust. In my opinion,
> we
> >>> >         need to fix that sooner rather than later.
> >>> >
> >>> >
> >>> >     Agree on the idea of focussing on stability instead of feature
> set
> >>> >     when it comes to LTS releases. Based on the previous discussion
> on
> >>> >     this [1] looks like the intended audience is enterprise customers
> >>> >     that would like to maintain a stable environment and that usually
> >>> >     have a long testing cycle before deploying a new version. Seems
> >>> >     like, rushing in features  or dependency upgrades for an LTS
> release
> >>> >     will be counterintuitive for such use cases.
> >>> >
> >>> >     On the other hand, I think a part of Ismaël point was that all
> major
> >>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
> >>> >     should be supportable for the duration of the time Beam LTS
> release
> >>> >     is supported (1 year based on [1]). If a major dependency is
> >>> >     expected to become unsupported within an year, it makes sense to
> >>> >     upgrade that before the LTS release. I suggest we do this at
> least
> >>> >     one release before the planned LTS release (so 2.9.0 if we want
> to
> >>> >     make 2.10.0 a LTS release) so that we have all major
> >>> >     dependency/feature updates battle tested before committing to
> them.
> >>> >
> >>> >
> >>> > Dependencies are a significant concern. What happens if one of our
> >>> > dependencies has no version they're going to support for a year
> hence?
> >>> > It could happen that there's never a time when all our dependencies
> will
> >>> > be supported for at least a year as of any given date. And rushing to
> >>> > get the "latest" version of a dependency in is often contrary to the
> >>> > goals of an LTS. (Better to have those depend on the battle-hardened
> >>> > versions.)
> >>> >
> >>> > The long-term goal is probably to avoid pinning specific versions as
> >>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever
> the
> >>> > user provided), and considered backporting support for 1.7 when it
> came
> >>> > out, we'd be in much better shape here. How feasible this is depends
> on
> >>> > how backwards-compatible the dependency is.
> >>> >
> >>> > On the other hand, forcing a user to upgrade a (user-visible, which
> in
> >>> > particular includes runners) dependency seems contrary to the goals
> of
> >>> > an LTS. Even if that dependency becomes unsupported.
> >>> >
> >>> >
> >>>
> >>> --
> >>> Jean-Baptiste Onofré
> >>> jbonofre@apache.org
> >>> http://blog.nanthrax.net
> >>> Talend - http://www.talend.com
> >>
> >>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ismaël Mejía <ie...@gmail.com>]

The simplest thing we can do is just to pin all the deps of the LTS
and not move them in any maintenance release if not a strong reason to
do so.

The next subject is to make maintainers aware of which release will be
the LTS in advance so they decide what to do with the dependencies
versions. In my previous mail I mentioned all the possible cases that
can happen with dependencies and it is clear that one unified policy
won’t satisfy every one. So better let the maintainers (who can also
ask for user feedback in the ML) to decide about  versions before the
release.

Alexey’s question is still a really important issue, and has been so
far ignored. What happens with the ‘Experimental’ APIs in the LTS.
Options are:

(1) We keep consistent with Experimental which means that they are
still not guarantees (note that this does not mean that they will be
broken arbitrarily).
(2) We are consistent with the LTS approach which makes them ‘non
experimental’ for the LTS so we will guarantee the functionality/API
stable.

I personally have conflicted opinions I would like to favor (1) but
this is not consistent with the whole idea of LTS so probably (2) is
wiser.

Finally I also worry about Tim’s remarks on performance and quality,
even if some of these things effectively can be fixed in a subsequent
LTS release. Users will probably prefer a LTS to start with Beam and
if the performance/quality of the LTS, this can hurt perception of the
project.
On Wed, Oct 10, 2018 at 4:53 AM Kenneth Knowles <ke...@apache.org> wrote:
>
> I've seen two mentions that "rushing" is contrary to the goals of LTS. But I wouldn't worry about this. The fact is there is almost nothing you can do to stabilize *prior* to cutting the LTS branch. Stability comes from the branch being long-lived and having multiple releases.
>
> (I think this is pretty much my version of what JB is saying)
>
> What a conservative user will do if 2.8.x is declared LTS is to start using the 2.8.x branch after it has had a couple bugfix releases. I don't think it is useful or possible to try for an "extra stable" 2.x.0.
>
> The arguments about supporting the most widely used versions of runner backends apply regardless of LTS. We should support them if we have the resources to do so.
>
> Kenn
>
> On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:
>>
>>
>>
>> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>>
>>> Hi,
>>>
>>> I think we have to remember what it's a LTS. A LTS is clearly a branch
>>> that we guarantee to have fixes on it for a long period of time.
>>>
>>>
>>> It doesn't mean that LTS == unique release. We can do a bunch of
>>> releases on a LTS branch, the only constraint is to avoid to introduce
>>> breaking changes.
>>
>>
>> I agree with this perspective. Thank you for sharing this. However the other commenters also had a good point. Requiring users to upgrade their runner version maybe incompatible with the goals of an LTS branch. Ideally the fixes here should be very minimal and targeted.
>>
>>>
>>>
>>> So, IMHO, the key part is not release, it's branch.
>>>
>>> The first thing to decide is the branch.
>>>
>>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
>>> branch. It's a branch where we will cherry-pick some important fixes in
>>> the future and where we will cut release. It's the approach I use in
>>> other Apache projects (especially Karaf) and it works fine.
>>
>>
>> JB, does Karaf has a documented process that we can re-use? If not could you explain a bit more?
>>
>> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0 release out of that?
>>
>>>
>>>
>>> Just my $0.01
>>>
>>> Regards
>>> JB
>>>
>>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <chamikara@google.com
>>> > <ma...@google.com>> wrote:
>>> >
>>> >
>>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>>> >     <ma...@google.com>> wrote:
>>> >
>>> >         I agree that LTS releases require more thought. Thank you for
>>> >         raising these questions. What other open questions do we have
>>> >         related LTS releases?
>>> >
>>> >         One way to do this would be to add them to a particular tracking
>>> >         list (e.g. 2.9.0 blocking list) that way we would be ready for
>>> >         an LTS release ahead of time.
>>> >
>>> >         Related to dependencies, I agree with Thomas. If we block on
>>> >         waiting for dependencies, we may end up taking a long time
>>> >         before making any LTS release. And the reality of Beam releases
>>> >         right now is that there are no supported releases today that and
>>> >         in the long term that might hurt user trust. In my opinion, we
>>> >         need to fix that sooner rather than later.
>>> >
>>> >
>>> >     Agree on the idea of focussing on stability instead of feature set
>>> >     when it comes to LTS releases. Based on the previous discussion on
>>> >     this [1] looks like the intended audience is enterprise customers
>>> >     that would like to maintain a stable environment and that usually
>>> >     have a long testing cycle before deploying a new version. Seems
>>> >     like, rushing in features  or dependency upgrades for an LTS release
>>> >     will be counterintuitive for such use cases.
>>> >
>>> >     On the other hand, I think a part of Ismaël point was that all major
>>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
>>> >     should be supportable for the duration of the time Beam LTS release
>>> >     is supported (1 year based on [1]). If a major dependency is
>>> >     expected to become unsupported within an year, it makes sense to
>>> >     upgrade that before the LTS release. I suggest we do this at least
>>> >     one release before the planned LTS release (so 2.9.0 if we want to
>>> >     make 2.10.0 a LTS release) so that we have all major
>>> >     dependency/feature updates battle tested before committing to them.
>>> >
>>> >
>>> > Dependencies are a significant concern. What happens if one of our
>>> > dependencies has no version they're going to support for a year hence?
>>> > It could happen that there's never a time when all our dependencies will
>>> > be supported for at least a year as of any given date. And rushing to
>>> > get the "latest" version of a dependency in is often contrary to the
>>> > goals of an LTS. (Better to have those depend on the battle-hardened
>>> > versions.)
>>> >
>>> > The long-term goal is probably to avoid pinning specific versions as
>>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever the
>>> > user provided), and considered backporting support for 1.7 when it came
>>> > out, we'd be in much better shape here. How feasible this is depends on
>>> > how backwards-compatible the dependency is.
>>> >
>>> > On the other hand, forcing a user to upgrade a (user-visible, which in
>>> > particular includes runners) dependency seems contrary to the goals of
>>> > an LTS. Even if that dependency becomes unsupported.
>>> >
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>
>>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Kenneth Knowles <ke...@apache.org>]

I've seen two mentions that "rushing" is contrary to the goals of LTS. But
I wouldn't worry about this. The fact is there is almost nothing you can do
to stabilize ***prior*** to cutting the LTS branch. Stability comes from
the branch being long-lived and having multiple releases.

(I think this is pretty much my version of what JB is saying)

What a conservative user will do if 2.8.x is declared LTS is to start using
the 2.8.x branch after it has had a couple bugfix releases. I don't think
it is useful or possible to try for an "extra stable" 2.x.0.

The arguments about supporting the most widely used versions of runner
backends apply regardless of LTS. We should support them if we have the
resources to do so.

Kenn

On Tue, Oct 9, 2018 at 4:57 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi,
>>
>> I think we have to remember what it's a LTS. A LTS is clearly a branch
>> that we guarantee to have fixes on it for a long period of time.
>
>
>> It doesn't mean that LTS == unique release. We can do a bunch of
>> releases on a LTS branch, the only constraint is to avoid to introduce
>> breaking changes.
>>
>
> I agree with this perspective. Thank you for sharing this. However the
> other commenters also had a good point. Requiring users to upgrade their
> runner version maybe incompatible with the goals of an LTS branch. Ideally
> the fixes here should be very minimal and targeted.
>
>
>>
>> So, IMHO, the key part is not release, it's branch.
>>
>> The first thing to decide is the branch.
>>
>> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
>> branch. It's a branch where we will cherry-pick some important fixes in
>> the future and where we will cut release. It's the approach I use in
>> other Apache projects (especially Karaf) and it works fine.
>>
>
> JB, does Karaf has a documented process that we can re-use? If not could
> you explain a bit more?
>
> Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0 release
> out of that?
>
>
>>
>> Just my $0.01
>>
>> Regards
>> JB
>>
>> On 05/10/2018 12:14, Robert Bradshaw wrote:
>> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <chamikara@google.com
>> > <ma...@google.com>> wrote:
>> >
>> >
>> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>> >     <ma...@google.com>> wrote:
>> >
>> >         I agree that LTS releases require more thought. Thank you for
>> >         raising these questions. What other open questions do we have
>> >         related LTS releases?
>> >
>> >         One way to do this would be to add them to a particular tracking
>> >         list (e.g. 2.9.0 blocking list) that way we would be ready for
>> >         an LTS release ahead of time.
>> >
>> >         Related to dependencies, I agree with Thomas. If we block on
>> >         waiting for dependencies, we may end up taking a long time
>> >         before making any LTS release. And the reality of Beam releases
>> >         right now is that there are no supported releases today that and
>> >         in the long term that might hurt user trust. In my opinion, we
>> >         need to fix that sooner rather than later.
>> >
>> >
>> >     Agree on the idea of focussing on stability instead of feature set
>> >     when it comes to LTS releases. Based on the previous discussion on
>> >     this [1] looks like the intended audience is enterprise customers
>> >     that would like to maintain a stable environment and that usually
>> >     have a long testing cycle before deploying a new version. Seems
>> >     like, rushing in features  or dependency upgrades for an LTS release
>> >     will be counterintuitive for such use cases.
>> >
>> >     On the other hand, I think a part of Ismaël point was that all major
>> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
>> >     should be supportable for the duration of the time Beam LTS release
>> >     is supported (1 year based on [1]). If a major dependency is
>> >     expected to become unsupported within an year, it makes sense to
>> >     upgrade that before the LTS release. I suggest we do this at least
>> >     one release before the planned LTS release (so 2.9.0 if we want to
>> >     make 2.10.0 a LTS release) so that we have all major
>> >     dependency/feature updates battle tested before committing to them.
>> >
>> >
>> > Dependencies are a significant concern. What happens if one of our
>> > dependencies has no version they're going to support for a year hence?
>> > It could happen that there's never a time when all our dependencies will
>> > be supported for at least a year as of any given date. And rushing to
>> > get the "latest" version of a dependency in is often contrary to the
>> > goals of an LTS. (Better to have those depend on the battle-hardened
>> > versions.)
>> >
>> > The long-term goal is probably to avoid pinning specific versions as
>> > much as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever the
>> > user provided), and considered backporting support for 1.7 when it came
>> > out, we'd be in much better shape here. How feasible this is depends on
>> > how backwards-compatible the dependency is.
>> >
>> > On the other hand, forcing a user to upgrade a (user-visible, which in
>> > particular includes runners) dependency seems contrary to the goals of
>> > an LTS. Even if that dependency becomes unsupported.
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Ahmet Altay <al...@google.com>]

On Fri, Oct 5, 2018 at 4:38 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:

> Hi,
>
> I think we have to remember what it's a LTS. A LTS is clearly a branch
> that we guarantee to have fixes on it for a long period of time.


> It doesn't mean that LTS == unique release. We can do a bunch of
> releases on a LTS branch, the only constraint is to avoid to introduce
> breaking changes.
>

I agree with this perspective. Thank you for sharing this. However the
other commenters also had a good point. Requiring users to upgrade their
runner version maybe incompatible with the goals of an LTS branch. Ideally
the fixes here should be very minimal and targeted.


>
> So, IMHO, the key part is not release, it's branch.
>
> The first thing to decide is the branch.
>
> Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
> branch. It's a branch where we will cherry-pick some important fixes in
> the future and where we will cut release. It's the approach I use in
> other Apache projects (especially Karaf) and it works fine.
>

JB, does Karaf has a documented process that we can re-use? If not could
you explain a bit more?

Is the proposal here to prepare 2.8.x LTS branch and make a 2.8.0 release
out of that?


>
> Just my $0.01
>
> Regards
> JB
>
> On 05/10/2018 12:14, Robert Bradshaw wrote:
> > On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <chamikara@google.com
> > <ma...@google.com>> wrote:
> >
> >
> >     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
> >     <ma...@google.com>> wrote:
> >
> >         I agree that LTS releases require more thought. Thank you for
> >         raising these questions. What other open questions do we have
> >         related LTS releases?
> >
> >         One way to do this would be to add them to a particular tracking
> >         list (e.g. 2.9.0 blocking list) that way we would be ready for
> >         an LTS release ahead of time.
> >
> >         Related to dependencies, I agree with Thomas. If we block on
> >         waiting for dependencies, we may end up taking a long time
> >         before making any LTS release. And the reality of Beam releases
> >         right now is that there are no supported releases today that and
> >         in the long term that might hurt user trust. In my opinion, we
> >         need to fix that sooner rather than later.
> >
> >
> >     Agree on the idea of focussing on stability instead of feature set
> >     when it comes to LTS releases. Based on the previous discussion on
> >     this [1] looks like the intended audience is enterprise customers
> >     that would like to maintain a stable environment and that usually
> >     have a long testing cycle before deploying a new version. Seems
> >     like, rushing in features  or dependency upgrades for an LTS release
> >     will be counterintuitive for such use cases.
> >
> >     On the other hand, I think a part of Ismaël point was that all major
> >     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
> >     should be supportable for the duration of the time Beam LTS release
> >     is supported (1 year based on [1]). If a major dependency is
> >     expected to become unsupported within an year, it makes sense to
> >     upgrade that before the LTS release. I suggest we do this at least
> >     one release before the planned LTS release (so 2.9.0 if we want to
> >     make 2.10.0 a LTS release) so that we have all major
> >     dependency/feature updates battle tested before committing to them.
> >
> >
> > Dependencies are a significant concern. What happens if one of our
> > dependencies has no version they're going to support for a year hence?
> > It could happen that there's never a time when all our dependencies will
> > be supported for at least a year as of any given date. And rushing to
> > get the "latest" version of a dependency in is often contrary to the
> > goals of an LTS. (Better to have those depend on the battle-hardened
> > versions.)
> >
> > The long-term goal is probably to avoid pinning specific versions as
> > much as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever the
> > user provided), and considered backporting support for 1.7 when it came
> > out, we'd be in much better shape here. How feasible this is depends on
> > how backwards-compatible the dependency is.
> >
> > On the other hand, forcing a user to upgrade a (user-visible, which in
> > particular includes runners) dependency seems contrary to the goals of
> > an LTS. Even if that dependency becomes unsupported.
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Hi,

I think we have to remember what it's a LTS. A LTS is clearly a branch
that we guarantee to have fixes on it for a long period of time.

It doesn't mean that LTS == unique release. We can do a bunch of
releases on a LTS branch, the only constraint is to avoid to introduce
breaking changes.

So, IMHO, the key part is not release, it's branch.

The first thing to decide is the branch.

Instead of talking about 2.8.0 or 2.9.0, I would prepare a 2.8.x LTS
branch. It's a branch where we will cherry-pick some important fixes in
the future and where we will cut release. It's the approach I use in
other Apache projects (especially Karaf) and it works fine.

Just my $0.01

Regards
JB

On 05/10/2018 12:14, Robert Bradshaw wrote:
> On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <chamikara@google.com
> <ma...@google.com>> wrote:
> 
> 
>     On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <altay@google.com
>     <ma...@google.com>> wrote:
> 
>         I agree that LTS releases require more thought. Thank you for
>         raising these questions. What other open questions do we have
>         related LTS releases?
> 
>         One way to do this would be to add them to a particular tracking
>         list (e.g. 2.9.0 blocking list) that way we would be ready for
>         an LTS release ahead of time.
> 
>         Related to dependencies, I agree with Thomas. If we block on
>         waiting for dependencies, we may end up taking a long time
>         before making any LTS release. And the reality of Beam releases
>         right now is that there are no supported releases today that and
>         in the long term that might hurt user trust. In my opinion, we
>         need to fix that sooner rather than later.
> 
> 
>     Agree on the idea of focussing on stability instead of feature set
>     when it comes to LTS releases. Based on the previous discussion on
>     this [1] looks like the intended audience is enterprise customers
>     that would like to maintain a stable environment and that usually
>     have a long testing cycle before deploying a new version. Seems
>     like, rushing in features  or dependency upgrades for an LTS release
>     will be counterintuitive for such use cases.
> 
>     On the other hand, I think a part of Ismaël point was that all major
>     features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..)
>     should be supportable for the duration of the time Beam LTS release
>     is supported (1 year based on [1]). If a major dependency is
>     expected to become unsupported within an year, it makes sense to
>     upgrade that before the LTS release. I suggest we do this at least
>     one release before the planned LTS release (so 2.9.0 if we want to
>     make 2.10.0 a LTS release) so that we have all major
>     dependency/feature updates battle tested before committing to them.
> 
> 
> Dependencies are a significant concern. What happens if one of our
> dependencies has no version they're going to support for a year hence?
> It could happen that there's never a time when all our dependencies will
> be supported for at least a year as of any given date. And rushing to
> get the "latest" version of a dependency in is often contrary to the
> goals of an LTS. (Better to have those depend on the battle-hardened
> versions.)
> 
> The long-term goal is probably to avoid pinning specific versions as
> much as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever the
> user provided), and considered backporting support for 1.7 when it came
> out, we'd be in much better shape here. How feasible this is depends on
> how backwards-compatible the dependency is. 
> 
> On the other hand, forcing a user to upgrade a (user-visible, which in
> particular includes runners) dependency seems contrary to the goals of
> an LTS. Even if that dependency becomes unsupported. 
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Robert Bradshaw <ro...@google.com>]

On Fri, Oct 5, 2018 at 3:59 AM Chamikara Jayalath <ch...@google.com>
wrote:

>
> On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <al...@google.com> wrote:
>
>> I agree that LTS releases require more thought. Thank you for raising
>> these questions. What other open questions do we have related LTS releases?
>>
>> One way to do this would be to add them to a particular tracking list
>> (e.g. 2.9.0 blocking list) that way we would be ready for an LTS release
>> ahead of time.
>>
>> Related to dependencies, I agree with Thomas. If we block on waiting for
>> dependencies, we may end up taking a long time before making any LTS
>> release. And the reality of Beam releases right now is that there are no
>> supported releases today that and in the long term that might hurt user
>> trust. In my opinion, we need to fix that sooner rather than later.
>>
>
> Agree on the idea of focussing on stability instead of feature set when it
> comes to LTS releases. Based on the previous discussion on this [1] looks
> like the intended audience is enterprise customers that would like to
> maintain a stable environment and that usually have a long testing cycle
> before deploying a new version. Seems like, rushing in features  or
> dependency upgrades for an LTS release will be counterintuitive for such
> use cases.
>
> On the other hand, I think a part of Ismaël point was that all major
> features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..) should be
> supportable for the duration of the time Beam LTS release is supported (1
> year based on [1]). If a major dependency is expected to become unsupported
> within an year, it makes sense to upgrade that before the LTS release. I
> suggest we do this at least one release before the planned LTS release (so
> 2.9.0 if we want to make 2.10.0 a LTS release) so that we have all major
> dependency/feature updates battle tested before committing to them.
>

Dependencies are a significant concern. What happens if one of our
dependencies has no version they're going to support for a year hence? It
could happen that there's never a time when all our dependencies will be
supported for at least a year as of any given date. And rushing to get the
"latest" version of a dependency in is often contrary to the goals of an
LTS. (Better to have those depend on the battle-hardened versions.)

The long-term goal is probably to avoid pinning specific versions as much
as possible. E.g. if we worked with Flink 1.5 or 1.6 (whichever the user
provided), and considered backporting support for 1.7 when it came out,
we'd be in much better shape here. How feasible this is depends on how
backwards-compatible the dependency is.

On the other hand, forcing a user to upgrade a (user-visible, which in
particular includes runners) dependency seems contrary to the goals of an
LTS. Even if that dependency becomes unsupported.

Re: What is required for LTS releases? (was: [PROPOSAL] Prepare Beam 2.8.0 release)

[Chamikara Jayalath <ch...@google.com>]

On Thu, Oct 4, 2018 at 9:39 AM Ahmet Altay <al...@google.com> wrote:

> I agree that LTS releases require more thought. Thank you for raising
> these questions. What other open questions do we have related LTS releases?
>
> One way to do this would be to add them to a particular tracking list
> (e.g. 2.9.0 blocking list) that way we would be ready for an LTS release
> ahead of time.
>
> Related to dependencies, I agree with Thomas. If we block on waiting for
> dependencies, we may end up taking a long time before making any LTS
> release. And the reality of Beam releases right now is that there are no
> supported releases today that and in the long term that might hurt user
> trust. In my opinion, we need to fix that sooner rather than later.
>

Agree on the idea of focussing on stability instead of feature set when it
comes to LTS releases. Based on the previous discussion on this [1] looks
like the intended audience is enterprise customers that would like to
maintain a stable environment and that usually have a long testing cycle
before deploying a new version. Seems like, rushing in features  or
dependency upgrades for an LTS release will be counterintuitive for such
use cases.

On the other hand, I think a part of Ismaël point was that all major
features of Beam (Spark Runner, Flink Runner, Kafka IO, etc ..) should be
supportable for the duration of the time Beam LTS release is supported (1
year based on [1]). If a major dependency is expected to become unsupported
within an year, it makes sense to upgrade that before the LTS release. I
suggest we do this at least one release before the planned LTS release (so
2.9.0 if we want to make 2.10.0 a LTS release) so that we have all major
dependency/feature updates battle tested before committing to them.

Thanks,
Cham

 [1]
https://lists.apache.org/thread.html/b2e2e79e0e146ed5264a12831d9e40cabfcb75651a994484c58a6b01@%3Cdev.beam.apache.org%3E


>
> Ahmet
>
>
> On Thu, Oct 4, 2018 at 9:27 AM, Ahmet Altay <al...@google.com> wrote:
>
>> Thank you all for the feedback. I will continue with 2.8.0 as a regular
>> release and separate the LTS discussion to a new thread.
>>
>> On Thu, Oct 4, 2018 at 7:58 AM, Thomas Weise <th...@apache.org> wrote:
>>
>>> Given the feedback so far, we should probably decouple LTS and 2.8.0
>>> discussions. In case both converge before 10/10 then fine, but not
>>> necessary. I also agree that we should not jump the gun on LTS and minimum
>>> 72 hours feedback window for the topic looks appropriate.
>>>
>>> The issues raised by Tim look like blockers and unless we are confident
>>> that they can be addressed as a patch release may warrant to defer LTS? Can
>>> we start to tag such JIRAs with an LTS label?
>>>
>>> On the other hand, I think we could allow for a bit of experimentation
>>> error for the first LTS attempt and feed guidelines/policies from
>>> learnings/feedback.
>>>
>>> Dependency updates for LTS: I don't think we should block LTS because
>>> there is a newer version of a dependency out there or we should rush
>>> updates. If we prioritize stability, then the latest usually isn't the
>>> best. In the case of Flink, 1.5.x is probably what most users have at this
>>> time and it has seen 4 patch releases. If Flink community continues to
>>> support last two minor (X.Y) versions, then 1.5.x support may drop when 1.7
>>> comes out, but that does not mean we cannot use it if we were to cut a Beam
>>> LTS release today. I generally think that LTS needs to focus more on the
>>> stability of Beam itself.
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>>
>>> On Thu, Oct 4, 2018 at 6:59 AM Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>>
>>>> Regarding LTS release - I agree that we need to have clear view what
>>>> kind of support will be provided for such releases.
>>>>
>>>> Despite of the concerns mentioned before, I have another one about API
>>>> labeled as “@Experimental". I think there are most of IOs, SQL, PCollection
>>>> with Schema, etc, labeled with this annotation.
>>>> According to definition, such API should be considered as unstable in
>>>> terms that it can be changed/removed in next releases.
>>>>
>>>> So, the question is - how “@Experimental” API affects LTS releases (if
>>>> it does)? What kind of support should be provided in this case, especially,
>>>> in case if API continued evolving after LTS has been issued? Do we need to
>>>> provide a guarantee (another annotation, for example) that API won’t be
>>>> changed between two LTS releases?
>>>>
>>>> And one more related question, which probably deserves another
>>>> discussion (or was already discussed) - what is criteria to remove
>>>> status “@Experimental” from API? How we decide that API is stable and not
>>>> changing anymore?
>>>>
>>>>
>>>> On 4 Oct 2018, at 12:35, Robert Bradshaw <ro...@google.com> wrote:
>>>>
>>>> +1 to cutting the release.
>>>>
>>>> I agree that the LTS label requires more discussion. I think it boils
>>>> down to the question of whether we are comfortable with encouraging people
>>>> to not upgrade to the latest Beam. It probably boils down to creating a
>>>> list of (potential) blockers and then going from there. Also, on this note,
>>>> I think we should be very conservative in updating dependencies for an LTS
>>>> release.
>>>>
>>>> We could also consider for this release doing an "LTS light" where we
>>>> prove the process, gain some experience, but don't promise a full 12 months
>>>> of support (say, cutting it to 6 months).
>>>>
>>>> - Robert
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Oct 4, 2018 at 11:25 AM Tim Robertson <
>>>> timrobertson100@gmail.com> wrote:
>>>>
>>>>> I was in the middle of writing something similar when Ismaël posted.
>>>>>
>>>>> Please do bear in mind that this is an international project and 7hrs
>>>>> is not long enough to decide upon something that affects us all.
>>>>>
>>>>> +1 on cutting 2.8.0 on 10/10 and thank you for pushing it forward
>>>>>
>>>>> -1 on designating it as LTS:
>>>>> While LTS is a statement of expectation in maintenance it also carries
>>>>> an element of trust. I propose we should have a separate discussion about
>>>>> what we might like to collectively achieve before announcing our first LTS
>>>>> edition.
>>>>> My concern stems from usability and first impressions - for example:
>>>>> - Beam has real issues with HDFS today (BEAM-5036) which I propose as
>>>>> blocker for announcing LTS
>>>>> - DirectRunner and the inability to run basic pipelines on a few GB of
>>>>> data is *really* putting people off our project - we might consider
>>>>> exploring that as it affects our "brand"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Oct 4, 2018 at 11:18 AM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Thanks Ahmet for volunteering to do the release, and proposing this
>>>>>> as an LTS.
>>>>>>
>>>>>> I have still some questions on our LTS policies (which may have
>>>>>> consequences on the discussed release):
>>>>>>
>>>>>> What are the expected implications of upgrades in the LTS, e.g. If a
>>>>>> connector let’s say Kafka is released using the 1.0 dependency, can it
>>>>>> be moved upwards in a LTS to version 2.0 or this will be considered a
>>>>>> breaking change and we should only move in minor versions. Will this
>>>>>> rule be more relaxed for example for all cloud based dependencies
>>>>>> (GCP, AWS) for example if a security issue or correctness/performance
>>>>>> improvement?
>>>>>>
>>>>>> Given that this will last for a year maybe we should raise some of the
>>>>>> dependencies to the latest versions. Following the recent discussion
>>>>>> on dependencies that cannot be ‘automatically’ updated because of end
>>>>>> user consequences, I still think about what we should do with
>>>>>> (probably related to the previous paragraph):
>>>>>>
>>>>>> - Should we move Flink then to 1.6.x considering that 1.5.x won’t be
>>>>>> maintained in less than 6 months.
>>>>>> - Should we wait and upgrade Spark into version 2.4.0 (which is being
>>>>>> voted at this moment but not released but could make sense for a LTS)
>>>>>> or just stay in 2.3.x. Spark is less of an issue because it is a
>>>>>> provided dep but still worth.
>>>>>> - Should we update the IO connectors dependencies to the latest stable
>>>>>> versions who aren’t, e.g. Elasticsearch, HBase,
>>>>>>
>>>>>> Of course the goal is not a last minute rush to do this so it fits in
>>>>>> the LTS release, but to see that for LTS we may consider the ‘lasting
>>>>>> consequences'.
>>>>>>
>>>>>> One last comment, next time we discuss a proposal please ensure that
>>>>>> we wait at least 24h to reach conclusions or proceed, otherwise this
>>>>>> will exclude opinions from people who are not in the right time zone
>>>>>> (this is the reason why votes last 72h to ensure that everyone may be
>>>>>> aware of what is been voted). This is not a mandatory requirement, but
>>>>>> agreeing on a LTS in 7h seems a bit short.
>>>>>> On Thu, Oct 4, 2018 at 1:36 AM Ahmet Altay <al...@google.com> wrote:
>>>>>> >
>>>>>> > Great. I will do the cut on 10/10.
>>>>>> >
>>>>>> > Let's start by triaging the open issues targeted for 2.8.0 [1]. If
>>>>>> you have any issues in this list please resolve them or move to the next
>>>>>> release. If you are aware of any critical issues please add to this list.
>>>>>> >
>>>>>> > Ahmet
>>>>>> >
>>>>>> > [1]
>>>>>> https://issues.apache.org/jira/browse/BEAM-5456?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%202.8.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>>>>>> >
>>>>>> > > +1 for the 2.7.0 release schedule. Thanks for volunteering. Do we
>>>>>> want a standing owner for the LTS branch (like the Linux kernel has) or
>>>>>> will we just take volunteers for each LTS release as they arise?
>>>>>> >
>>>>>> > We have not thought about this before. IMO, it is better to keep
>>>>>> things simple and use the same process (i.e. "we just take volunteers for
>>>>>> each LTS release as they arise") for patch releases in the future if/when
>>>>>> we happen to need those.
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Oct 3, 2018 at 1:21 PM, Thomas Weise <th...@apache.org>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> +1
>>>>>> >>
>>>>>> >> On Wed, Oct 3, 2018 at 12:33 PM Ted Yu <yu...@gmail.com>
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>> +1
>>>>>> >>>
>>>>>> >>> On Wed, Oct 3, 2018 at 9:52 AM Jean-Baptiste Onofré <
>>>>>> jb@nanthrax.net> wrote:
>>>>>> >>>>
>>>>>> >>>> +1
>>>>>> >>>>
>>>>>> >>>> but we have to be fast in release process. 2.7.0 took more than
>>>>>> 1 month
>>>>>> >>>> to be cut !
>>>>>> >>>>
>>>>>> >>>> If no blocker, we have to just move forward.
>>>>>> >
>>>>>> >
>>>>>> > +1
>>>>>> >
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> Regards
>>>>>> >>>> JB
>>>>>> >>>>
>>>>>> >>>> On 03/10/2018 18:25, Ahmet Altay wrote:
>>>>>> >>>> > Hi all,
>>>>>> >>>> >
>>>>>> >>>> > Release cut date for the next release is 10/10 according to
>>>>>> Beam release
>>>>>> >>>> > calendar [1]. Since the previous release is already mostly
>>>>>> wrapped up
>>>>>> >>>> > (modulo blog post), I would like to propose starting the next
>>>>>> release on
>>>>>> >>>> > time (10/10).
>>>>>> >>>> >
>>>>>> >>>> > Additionally I propose designating this release as the first
>>>>>> >>>> > long-term-support (LTS) release [2]. This should have no
>>>>>> impact on the
>>>>>> >>>> > release process, however it would mean that we commit to patch
>>>>>> this
>>>>>> >>>> > release for the next 12 months for major issues.
>>>>>> >>>> >
>>>>>> >>>> > I volunteer to perform this release.
>>>>>> >>>> >
>>>>>> >>>> > What do you think?
>>>>>> >>>> >
>>>>>> >>>> > Ahmet
>>>>>> >>>> >
>>>>>> >>>> > [1]
>>>>>> https://calendar.google.com/calendar/embed?src=0p73sl034k80oob7seouanigd0%40group.calendar.google.com&ctz=America%2FLos_Angeles
>>>>>> >>>> > [2] https://beam.apache.org/community/policies/#releases
>>>>>> >>>>
>>>>>> >>>> --
>>>>>> >>>> Jean-Baptiste Onofré
>>>>>> >>>> jbonofre@apache.org
>>>>>> >>>> http://blog.nanthrax.net
>>>>>> >>>> Talend - http://www.talend.com
>>>>>> >
>>>>>> >
>>>>>>
>>>>>
>>>>
>>
>