You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Till Rohrmann <tr...@apache.org> on 2021/12/02 14:57:28 UTC
[DISCUSS] FLIP-197: API stability graduation process
Hi everyone,
As promised, here is the follow-up FLIP [1] for discussing how we can
ensure that newly introduced APIs are being stabilized over time. This FLIP
is related to FLIP-196 [2].
The idea of FLIP-197 is to introduce an API graduation process that forces
us to increase the API stability guarantee unless there is a very good
reason not to do so. So the proposal is to reverse the process from opt-in
(increasing the stability guarantee explicitly) to opt-out (deciding that
an API cannot be graduated with a good reason).
Since every process breaks if it is not automated, we propose a richer set
of API stability annotations that can capture enough information so that we
can implement a test that fails if we fail to follow the process.
Looking forward to your feedback.
Hopefully, we can provide our users a better experience when working with
Flink because we offer more stable APIs and make them available faster.
[1] https://cwiki.apache.org/confluence/x/J5eqCw
[2] https://cwiki.apache.org/confluence/x/IJeqCw
Cheers,
Till
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Till Rohrmann <tr...@apache.org>.
Thanks for the feedback Konstantin.
* I agree that if we think that one release cycle to graduate is too short,
then we should extend it to something we believe we can achieve. For
FLIP-27, I think we could have stabilized the API faster if it had been our
priority (not sure whether two release cycles would have been enough
though).
* REST API stabilization is unrelated to this FLIP. Hence, I would like to
exclude it from this discussion.
Cheers,
Till
On Fri, Dec 3, 2021 at 9:42 AM Konstantin Knauf <kn...@apache.org> wrote:
> Hi Till,
>
> This makes sense to me, in general. I am wondering two things:
>
> * is one release to promote enough or will it lead to a lot of exclusions?
> FLIP-27, for example, took much longer to stabilize, I believe. If we
> think, this is quite a strict rule, then in my experience, let's rather
> start with an easier rule that is broken less, then a strict rule that is
> broken from the start.
> * If I could choose whether we focus on this FLIP or another follow up of
> FLIP-196 like establishing stability guarantees for the REST API, I would
> choose the latter.
>
> Cheers,
>
> Konstantin
>
>
>
> On Thu, Dec 2, 2021 at 3:58 PM Till Rohrmann <tr...@apache.org> wrote:
>
> > Hi everyone,
> >
> > As promised, here is the follow-up FLIP [1] for discussing how we can
> > ensure that newly introduced APIs are being stabilized over time. This
> FLIP
> > is related to FLIP-196 [2].
> >
> > The idea of FLIP-197 is to introduce an API graduation process that
> forces
> > us to increase the API stability guarantee unless there is a very good
> > reason not to do so. So the proposal is to reverse the process from
> opt-in
> > (increasing the stability guarantee explicitly) to opt-out (deciding that
> > an API cannot be graduated with a good reason).
> >
> > Since every process breaks if it is not automated, we propose a richer
> set
> > of API stability annotations that can capture enough information so that
> we
> > can implement a test that fails if we fail to follow the process.
> >
> > Looking forward to your feedback.
> >
> > Hopefully, we can provide our users a better experience when working with
> > Flink because we offer more stable APIs and make them available faster.
> >
> > [1] https://cwiki.apache.org/confluence/x/J5eqCw
> > [2] https://cwiki.apache.org/confluence/x/IJeqCw
> >
> > Cheers,
> > Till
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Konstantin Knauf <kn...@apache.org>.
Hi Till,
This makes sense to me, in general. I am wondering two things:
* is one release to promote enough or will it lead to a lot of exclusions?
FLIP-27, for example, took much longer to stabilize, I believe. If we
think, this is quite a strict rule, then in my experience, let's rather
start with an easier rule that is broken less, then a strict rule that is
broken from the start.
* If I could choose whether we focus on this FLIP or another follow up of
FLIP-196 like establishing stability guarantees for the REST API, I would
choose the latter.
Cheers,
Konstantin
On Thu, Dec 2, 2021 at 3:58 PM Till Rohrmann <tr...@apache.org> wrote:
> Hi everyone,
>
> As promised, here is the follow-up FLIP [1] for discussing how we can
> ensure that newly introduced APIs are being stabilized over time. This FLIP
> is related to FLIP-196 [2].
>
> The idea of FLIP-197 is to introduce an API graduation process that forces
> us to increase the API stability guarantee unless there is a very good
> reason not to do so. So the proposal is to reverse the process from opt-in
> (increasing the stability guarantee explicitly) to opt-out (deciding that
> an API cannot be graduated with a good reason).
>
> Since every process breaks if it is not automated, we propose a richer set
> of API stability annotations that can capture enough information so that we
> can implement a test that fails if we fail to follow the process.
>
> Looking forward to your feedback.
>
> Hopefully, we can provide our users a better experience when working with
> Flink because we offer more stable APIs and make them available faster.
>
> [1] https://cwiki.apache.org/confluence/x/J5eqCw
> [2] https://cwiki.apache.org/confluence/x/IJeqCw
>
> Cheers,
> Till
>
--
Konstantin Knauf
https://twitter.com/snntrable
https://github.com/knaufk
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Till Rohrmann <tr...@apache.org>.
Hi everyone,
Thanks a lot for all your input. I hope that I could answer most of the
questions. I would now start a vote on this FLIP.
Cheers,
Till
On Tue, Dec 7, 2021 at 3:05 PM Till Rohrmann <tr...@apache.org> wrote:
> Hi Timo,
>
> 1) I haven't fully thought where to put it but I guess that flink-core
> might be a good place for the FlinkVersion. We should then also unify any
> other enums if possible.
>
> 2) I think it makes sense to also review our deprecation process. I think
> it makes a lot of sense to add since when something is deprecated and
> establish a process for removing it (e.g. after one release and if not then
> state why). For the sake of brevity of this FLIP, I would move this
> discussion to a follow-up FLIP, though.
>
> Cheers,
> Till
>
> On Mon, Dec 6, 2021 at 11:35 AM Timo Walther <tw...@apache.org> wrote:
>
>> Hi Till,
>>
>> thanks for starting this discussion. I think this topic should have been
>> discussed way earlier. I have two questions:
>>
>> 1) It might be an implementation detail but where do you expect this
>> `FlinkVersion` to be located? This is actually a quite important class
>> that also needs to be made available for other use cases. For the SQL
>> upgrade story we will definitely need a similar enum. And I think we
>> have something similar for other tests (see `MigrationVersion`). For
>> reducing releasing overhead, it would be good to unify all these
>> "version metadata".
>>
>> 2) Deprecations: Shall we also start versioning deprecation decisions?
>> Esp. for Experimental/PublicEvolving interfaces we should remove
>> deprecated code in time. We should also let users know when we are
>> planning to remove code. E.g. clearly indicate that the deprecation will
>> happen in the next major version?
>>
>> Regards,
>> Timo
>>
>> On 06.12.21 10:04, Till Rohrmann wrote:
>> > Ok, then lets increase the graduation period to 2 releases. If we see
>> that
>> > this is super easy for us to do, then we can shorten it in the future.
>> >
>> > Cheers,
>> > Till
>> >
>> > On Mon, Dec 6, 2021 at 9:54 AM Chesnay Schepler <ch...@apache.org>
>> wrote:
>> >
>> >> Given that you can delay the graduation if there is a good reason for
>> >> it, we should be able to cover that case even if the graduation would
>> >> happen by default after 1 month.
>> >>
>> >> That said, personally I would also be in favor of 2 releases; we see
>> >> plenty of users not upgrading to every single Flink version, and this
>> >> may give us a bit more coverage.
>> >>
>> >> On 06/12/2021 09:20, Ingo Bürk wrote:
>> >>> Hi Till,
>> >>>
>> >>> from my (admittedly limited) experience with how far projects lag
>> behind
>> >> in
>> >>> terms of Flink versions – yes, the combined time it would take to
>> mature
>> >>> then seems reasonable enough for a sufficient adoption, IMO.
>> >>>
>> >>> Another reason why I think two releases as a default for the last step
>> >>> makes sense: say you mature an API to PublicEvolving. Typically, there
>> >> will
>> >>> be issues found afterwards. Even if you address these in the very next
>> >>> release cycle, a duration of one release would mean you fully mature
>> the
>> >>> API in the same release in which things are still being fixed;
>> >> intuitively,
>> >>> it makes sense to me that the step to Public would come after a
>> period of
>> >>> no changes needed, however.
>> >>>
>> >>>
>> >>> Ingo
>> >>>
>> >>> On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org>
>> >> wrote:
>> >>>
>> >>>> Hi Ingo, thanks for your feedback.
>> >>>>
>> >>>> Do you think that two release cycles per graduation step would be
>> long
>> >>>> enough or should it be longer?
>> >>>>
>> >>>> Cheers,
>> >>>> Till
>> >>>>
>> >>>> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
>> >>>>
>> >>>>> Hi Till,
>> >>>>>
>> >>>>> Overall I whole-heartedly agree with the proposals in this FLIP.
>> Thank
>> >>>> you
>> >>>>> for starting this discussion as well! This seems like something that
>> >>>> could
>> >>>>> be tested quite nicely with ArchUnit as well; I'll be happy to help
>> >>>> should
>> >>>>> the FLIP be accepted.
>> >>>>>
>> >>>>>> I would propose per default a single release.
>> >>>>> The step from PublicEvolving to Public feels more important to me,
>> and
>> >> I
>> >>>>> would personally suggest making this transition a bit longer. We
>> have a
>> >>>> bit
>> >>>>> of a chicken-egg problem here, because the goal of your FLIP is,
>> >>>>> ultimately, also to motivate faster adoption of new Flink versions,
>> but
>> >>>> the
>> >>>>> status quo prevents that; if we mature APIs too quickly, we risk
>> losing
>> >>>> out
>> >>>>> on important feedback. Therefore, I would propose starting slower
>> here,
>> >>>> and
>> >>>>> rather think about shortening that cycle in the future.
>> >>>>>
>> >>>>>
>> >>>>> Best
>> >>>>> Ingo
>> >>>>>
>> >>>>> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
>> >>>> wrote:
>> >>>>>> Hi everyone,
>> >>>>>>
>> >>>>>> As promised, here is the follow-up FLIP [1] for discussing how we
>> can
>> >>>>>> ensure that newly introduced APIs are being stabilized over time.
>> This
>> >>>>> FLIP
>> >>>>>> is related to FLIP-196 [2].
>> >>>>>>
>> >>>>>> The idea of FLIP-197 is to introduce an API graduation process that
>> >>>>> forces
>> >>>>>> us to increase the API stability guarantee unless there is a very
>> good
>> >>>>>> reason not to do so. So the proposal is to reverse the process from
>> >>>>> opt-in
>> >>>>>> (increasing the stability guarantee explicitly) to opt-out
>> (deciding
>> >>>> that
>> >>>>>> an API cannot be graduated with a good reason).
>> >>>>>>
>> >>>>>> Since every process breaks if it is not automated, we propose a
>> richer
>> >>>>> set
>> >>>>>> of API stability annotations that can capture enough information so
>> >>>> that
>> >>>>> we
>> >>>>>> can implement a test that fails if we fail to follow the process.
>> >>>>>>
>> >>>>>> Looking forward to your feedback.
>> >>>>>>
>> >>>>>> Hopefully, we can provide our users a better experience when
>> working
>> >>>> with
>> >>>>>> Flink because we offer more stable APIs and make them available
>> >> faster.
>> >>>>>>
>> >>>>>> [1] https://cwiki.apache.org/confluence/x/J5eqCw
>> >>>>>> [2] https://cwiki.apache.org/confluence/x/IJeqCw
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Till
>> >>>>>>
>> >>
>> >>
>> >
>>
>>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Till Rohrmann <tr...@apache.org>.
Hi Timo,
1) I haven't fully thought where to put it but I guess that flink-core
might be a good place for the FlinkVersion. We should then also unify any
other enums if possible.
2) I think it makes sense to also review our deprecation process. I think
it makes a lot of sense to add since when something is deprecated and
establish a process for removing it (e.g. after one release and if not then
state why). For the sake of brevity of this FLIP, I would move this
discussion to a follow-up FLIP, though.
Cheers,
Till
On Mon, Dec 6, 2021 at 11:35 AM Timo Walther <tw...@apache.org> wrote:
> Hi Till,
>
> thanks for starting this discussion. I think this topic should have been
> discussed way earlier. I have two questions:
>
> 1) It might be an implementation detail but where do you expect this
> `FlinkVersion` to be located? This is actually a quite important class
> that also needs to be made available for other use cases. For the SQL
> upgrade story we will definitely need a similar enum. And I think we
> have something similar for other tests (see `MigrationVersion`). For
> reducing releasing overhead, it would be good to unify all these
> "version metadata".
>
> 2) Deprecations: Shall we also start versioning deprecation decisions?
> Esp. for Experimental/PublicEvolving interfaces we should remove
> deprecated code in time. We should also let users know when we are
> planning to remove code. E.g. clearly indicate that the deprecation will
> happen in the next major version?
>
> Regards,
> Timo
>
> On 06.12.21 10:04, Till Rohrmann wrote:
> > Ok, then lets increase the graduation period to 2 releases. If we see
> that
> > this is super easy for us to do, then we can shorten it in the future.
> >
> > Cheers,
> > Till
> >
> > On Mon, Dec 6, 2021 at 9:54 AM Chesnay Schepler <ch...@apache.org>
> wrote:
> >
> >> Given that you can delay the graduation if there is a good reason for
> >> it, we should be able to cover that case even if the graduation would
> >> happen by default after 1 month.
> >>
> >> That said, personally I would also be in favor of 2 releases; we see
> >> plenty of users not upgrading to every single Flink version, and this
> >> may give us a bit more coverage.
> >>
> >> On 06/12/2021 09:20, Ingo Bürk wrote:
> >>> Hi Till,
> >>>
> >>> from my (admittedly limited) experience with how far projects lag
> behind
> >> in
> >>> terms of Flink versions – yes, the combined time it would take to
> mature
> >>> then seems reasonable enough for a sufficient adoption, IMO.
> >>>
> >>> Another reason why I think two releases as a default for the last step
> >>> makes sense: say you mature an API to PublicEvolving. Typically, there
> >> will
> >>> be issues found afterwards. Even if you address these in the very next
> >>> release cycle, a duration of one release would mean you fully mature
> the
> >>> API in the same release in which things are still being fixed;
> >> intuitively,
> >>> it makes sense to me that the step to Public would come after a period
> of
> >>> no changes needed, however.
> >>>
> >>>
> >>> Ingo
> >>>
> >>> On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org>
> >> wrote:
> >>>
> >>>> Hi Ingo, thanks for your feedback.
> >>>>
> >>>> Do you think that two release cycles per graduation step would be long
> >>>> enough or should it be longer?
> >>>>
> >>>> Cheers,
> >>>> Till
> >>>>
> >>>> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
> >>>>
> >>>>> Hi Till,
> >>>>>
> >>>>> Overall I whole-heartedly agree with the proposals in this FLIP.
> Thank
> >>>> you
> >>>>> for starting this discussion as well! This seems like something that
> >>>> could
> >>>>> be tested quite nicely with ArchUnit as well; I'll be happy to help
> >>>> should
> >>>>> the FLIP be accepted.
> >>>>>
> >>>>>> I would propose per default a single release.
> >>>>> The step from PublicEvolving to Public feels more important to me,
> and
> >> I
> >>>>> would personally suggest making this transition a bit longer. We
> have a
> >>>> bit
> >>>>> of a chicken-egg problem here, because the goal of your FLIP is,
> >>>>> ultimately, also to motivate faster adoption of new Flink versions,
> but
> >>>> the
> >>>>> status quo prevents that; if we mature APIs too quickly, we risk
> losing
> >>>> out
> >>>>> on important feedback. Therefore, I would propose starting slower
> here,
> >>>> and
> >>>>> rather think about shortening that cycle in the future.
> >>>>>
> >>>>>
> >>>>> Best
> >>>>> Ingo
> >>>>>
> >>>>> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
> >>>> wrote:
> >>>>>> Hi everyone,
> >>>>>>
> >>>>>> As promised, here is the follow-up FLIP [1] for discussing how we
> can
> >>>>>> ensure that newly introduced APIs are being stabilized over time.
> This
> >>>>> FLIP
> >>>>>> is related to FLIP-196 [2].
> >>>>>>
> >>>>>> The idea of FLIP-197 is to introduce an API graduation process that
> >>>>> forces
> >>>>>> us to increase the API stability guarantee unless there is a very
> good
> >>>>>> reason not to do so. So the proposal is to reverse the process from
> >>>>> opt-in
> >>>>>> (increasing the stability guarantee explicitly) to opt-out (deciding
> >>>> that
> >>>>>> an API cannot be graduated with a good reason).
> >>>>>>
> >>>>>> Since every process breaks if it is not automated, we propose a
> richer
> >>>>> set
> >>>>>> of API stability annotations that can capture enough information so
> >>>> that
> >>>>> we
> >>>>>> can implement a test that fails if we fail to follow the process.
> >>>>>>
> >>>>>> Looking forward to your feedback.
> >>>>>>
> >>>>>> Hopefully, we can provide our users a better experience when working
> >>>> with
> >>>>>> Flink because we offer more stable APIs and make them available
> >> faster.
> >>>>>>
> >>>>>> [1] https://cwiki.apache.org/confluence/x/J5eqCw
> >>>>>> [2] https://cwiki.apache.org/confluence/x/IJeqCw
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Till
> >>>>>>
> >>
> >>
> >
>
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Timo Walther <tw...@apache.org>.
Hi Till,
thanks for starting this discussion. I think this topic should have been
discussed way earlier. I have two questions:
1) It might be an implementation detail but where do you expect this
`FlinkVersion` to be located? This is actually a quite important class
that also needs to be made available for other use cases. For the SQL
upgrade story we will definitely need a similar enum. And I think we
have something similar for other tests (see `MigrationVersion`). For
reducing releasing overhead, it would be good to unify all these
"version metadata".
2) Deprecations: Shall we also start versioning deprecation decisions?
Esp. for Experimental/PublicEvolving interfaces we should remove
deprecated code in time. We should also let users know when we are
planning to remove code. E.g. clearly indicate that the deprecation will
happen in the next major version?
Regards,
Timo
On 06.12.21 10:04, Till Rohrmann wrote:
> Ok, then lets increase the graduation period to 2 releases. If we see that
> this is super easy for us to do, then we can shorten it in the future.
>
> Cheers,
> Till
>
> On Mon, Dec 6, 2021 at 9:54 AM Chesnay Schepler <ch...@apache.org> wrote:
>
>> Given that you can delay the graduation if there is a good reason for
>> it, we should be able to cover that case even if the graduation would
>> happen by default after 1 month.
>>
>> That said, personally I would also be in favor of 2 releases; we see
>> plenty of users not upgrading to every single Flink version, and this
>> may give us a bit more coverage.
>>
>> On 06/12/2021 09:20, Ingo Bürk wrote:
>>> Hi Till,
>>>
>>> from my (admittedly limited) experience with how far projects lag behind
>> in
>>> terms of Flink versions – yes, the combined time it would take to mature
>>> then seems reasonable enough for a sufficient adoption, IMO.
>>>
>>> Another reason why I think two releases as a default for the last step
>>> makes sense: say you mature an API to PublicEvolving. Typically, there
>> will
>>> be issues found afterwards. Even if you address these in the very next
>>> release cycle, a duration of one release would mean you fully mature the
>>> API in the same release in which things are still being fixed;
>> intuitively,
>>> it makes sense to me that the step to Public would come after a period of
>>> no changes needed, however.
>>>
>>>
>>> Ingo
>>>
>>> On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org>
>> wrote:
>>>
>>>> Hi Ingo, thanks for your feedback.
>>>>
>>>> Do you think that two release cycles per graduation step would be long
>>>> enough or should it be longer?
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
>>>>
>>>>> Hi Till,
>>>>>
>>>>> Overall I whole-heartedly agree with the proposals in this FLIP. Thank
>>>> you
>>>>> for starting this discussion as well! This seems like something that
>>>> could
>>>>> be tested quite nicely with ArchUnit as well; I'll be happy to help
>>>> should
>>>>> the FLIP be accepted.
>>>>>
>>>>>> I would propose per default a single release.
>>>>> The step from PublicEvolving to Public feels more important to me, and
>> I
>>>>> would personally suggest making this transition a bit longer. We have a
>>>> bit
>>>>> of a chicken-egg problem here, because the goal of your FLIP is,
>>>>> ultimately, also to motivate faster adoption of new Flink versions, but
>>>> the
>>>>> status quo prevents that; if we mature APIs too quickly, we risk losing
>>>> out
>>>>> on important feedback. Therefore, I would propose starting slower here,
>>>> and
>>>>> rather think about shortening that cycle in the future.
>>>>>
>>>>>
>>>>> Best
>>>>> Ingo
>>>>>
>>>>> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
>>>> wrote:
>>>>>> Hi everyone,
>>>>>>
>>>>>> As promised, here is the follow-up FLIP [1] for discussing how we can
>>>>>> ensure that newly introduced APIs are being stabilized over time. This
>>>>> FLIP
>>>>>> is related to FLIP-196 [2].
>>>>>>
>>>>>> The idea of FLIP-197 is to introduce an API graduation process that
>>>>> forces
>>>>>> us to increase the API stability guarantee unless there is a very good
>>>>>> reason not to do so. So the proposal is to reverse the process from
>>>>> opt-in
>>>>>> (increasing the stability guarantee explicitly) to opt-out (deciding
>>>> that
>>>>>> an API cannot be graduated with a good reason).
>>>>>>
>>>>>> Since every process breaks if it is not automated, we propose a richer
>>>>> set
>>>>>> of API stability annotations that can capture enough information so
>>>> that
>>>>> we
>>>>>> can implement a test that fails if we fail to follow the process.
>>>>>>
>>>>>> Looking forward to your feedback.
>>>>>>
>>>>>> Hopefully, we can provide our users a better experience when working
>>>> with
>>>>>> Flink because we offer more stable APIs and make them available
>> faster.
>>>>>>
>>>>>> [1] https://cwiki.apache.org/confluence/x/J5eqCw
>>>>>> [2] https://cwiki.apache.org/confluence/x/IJeqCw
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>
>>
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Till Rohrmann <tr...@apache.org>.
Ok, then lets increase the graduation period to 2 releases. If we see that
this is super easy for us to do, then we can shorten it in the future.
Cheers,
Till
On Mon, Dec 6, 2021 at 9:54 AM Chesnay Schepler <ch...@apache.org> wrote:
> Given that you can delay the graduation if there is a good reason for
> it, we should be able to cover that case even if the graduation would
> happen by default after 1 month.
>
> That said, personally I would also be in favor of 2 releases; we see
> plenty of users not upgrading to every single Flink version, and this
> may give us a bit more coverage.
>
> On 06/12/2021 09:20, Ingo Bürk wrote:
> > Hi Till,
> >
> > from my (admittedly limited) experience with how far projects lag behind
> in
> > terms of Flink versions – yes, the combined time it would take to mature
> > then seems reasonable enough for a sufficient adoption, IMO.
> >
> > Another reason why I think two releases as a default for the last step
> > makes sense: say you mature an API to PublicEvolving. Typically, there
> will
> > be issues found afterwards. Even if you address these in the very next
> > release cycle, a duration of one release would mean you fully mature the
> > API in the same release in which things are still being fixed;
> intuitively,
> > it makes sense to me that the step to Public would come after a period of
> > no changes needed, however.
> >
> >
> > Ingo
> >
> > On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> >> Hi Ingo, thanks for your feedback.
> >>
> >> Do you think that two release cycles per graduation step would be long
> >> enough or should it be longer?
> >>
> >> Cheers,
> >> Till
> >>
> >> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
> >>
> >>> Hi Till,
> >>>
> >>> Overall I whole-heartedly agree with the proposals in this FLIP. Thank
> >> you
> >>> for starting this discussion as well! This seems like something that
> >> could
> >>> be tested quite nicely with ArchUnit as well; I'll be happy to help
> >> should
> >>> the FLIP be accepted.
> >>>
> >>>> I would propose per default a single release.
> >>> The step from PublicEvolving to Public feels more important to me, and
> I
> >>> would personally suggest making this transition a bit longer. We have a
> >> bit
> >>> of a chicken-egg problem here, because the goal of your FLIP is,
> >>> ultimately, also to motivate faster adoption of new Flink versions, but
> >> the
> >>> status quo prevents that; if we mature APIs too quickly, we risk losing
> >> out
> >>> on important feedback. Therefore, I would propose starting slower here,
> >> and
> >>> rather think about shortening that cycle in the future.
> >>>
> >>>
> >>> Best
> >>> Ingo
> >>>
> >>> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
> >> wrote:
> >>>> Hi everyone,
> >>>>
> >>>> As promised, here is the follow-up FLIP [1] for discussing how we can
> >>>> ensure that newly introduced APIs are being stabilized over time. This
> >>> FLIP
> >>>> is related to FLIP-196 [2].
> >>>>
> >>>> The idea of FLIP-197 is to introduce an API graduation process that
> >>> forces
> >>>> us to increase the API stability guarantee unless there is a very good
> >>>> reason not to do so. So the proposal is to reverse the process from
> >>> opt-in
> >>>> (increasing the stability guarantee explicitly) to opt-out (deciding
> >> that
> >>>> an API cannot be graduated with a good reason).
> >>>>
> >>>> Since every process breaks if it is not automated, we propose a richer
> >>> set
> >>>> of API stability annotations that can capture enough information so
> >> that
> >>> we
> >>>> can implement a test that fails if we fail to follow the process.
> >>>>
> >>>> Looking forward to your feedback.
> >>>>
> >>>> Hopefully, we can provide our users a better experience when working
> >> with
> >>>> Flink because we offer more stable APIs and make them available
> faster.
> >>>>
> >>>> [1] https://cwiki.apache.org/confluence/x/J5eqCw
> >>>> [2] https://cwiki.apache.org/confluence/x/IJeqCw
> >>>>
> >>>> Cheers,
> >>>> Till
> >>>>
>
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Chesnay Schepler <ch...@apache.org>.
Given that you can delay the graduation if there is a good reason for
it, we should be able to cover that case even if the graduation would
happen by default after 1 month.
That said, personally I would also be in favor of 2 releases; we see
plenty of users not upgrading to every single Flink version, and this
may give us a bit more coverage.
On 06/12/2021 09:20, Ingo Bürk wrote:
> Hi Till,
>
> from my (admittedly limited) experience with how far projects lag behind in
> terms of Flink versions – yes, the combined time it would take to mature
> then seems reasonable enough for a sufficient adoption, IMO.
>
> Another reason why I think two releases as a default for the last step
> makes sense: say you mature an API to PublicEvolving. Typically, there will
> be issues found afterwards. Even if you address these in the very next
> release cycle, a duration of one release would mean you fully mature the
> API in the same release in which things are still being fixed; intuitively,
> it makes sense to me that the step to Public would come after a period of
> no changes needed, however.
>
>
> Ingo
>
> On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org> wrote:
>
>> Hi Ingo, thanks for your feedback.
>>
>> Do you think that two release cycles per graduation step would be long
>> enough or should it be longer?
>>
>> Cheers,
>> Till
>>
>> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
>>
>>> Hi Till,
>>>
>>> Overall I whole-heartedly agree with the proposals in this FLIP. Thank
>> you
>>> for starting this discussion as well! This seems like something that
>> could
>>> be tested quite nicely with ArchUnit as well; I'll be happy to help
>> should
>>> the FLIP be accepted.
>>>
>>>> I would propose per default a single release.
>>> The step from PublicEvolving to Public feels more important to me, and I
>>> would personally suggest making this transition a bit longer. We have a
>> bit
>>> of a chicken-egg problem here, because the goal of your FLIP is,
>>> ultimately, also to motivate faster adoption of new Flink versions, but
>> the
>>> status quo prevents that; if we mature APIs too quickly, we risk losing
>> out
>>> on important feedback. Therefore, I would propose starting slower here,
>> and
>>> rather think about shortening that cycle in the future.
>>>
>>>
>>> Best
>>> Ingo
>>>
>>> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
>> wrote:
>>>> Hi everyone,
>>>>
>>>> As promised, here is the follow-up FLIP [1] for discussing how we can
>>>> ensure that newly introduced APIs are being stabilized over time. This
>>> FLIP
>>>> is related to FLIP-196 [2].
>>>>
>>>> The idea of FLIP-197 is to introduce an API graduation process that
>>> forces
>>>> us to increase the API stability guarantee unless there is a very good
>>>> reason not to do so. So the proposal is to reverse the process from
>>> opt-in
>>>> (increasing the stability guarantee explicitly) to opt-out (deciding
>> that
>>>> an API cannot be graduated with a good reason).
>>>>
>>>> Since every process breaks if it is not automated, we propose a richer
>>> set
>>>> of API stability annotations that can capture enough information so
>> that
>>> we
>>>> can implement a test that fails if we fail to follow the process.
>>>>
>>>> Looking forward to your feedback.
>>>>
>>>> Hopefully, we can provide our users a better experience when working
>> with
>>>> Flink because we offer more stable APIs and make them available faster.
>>>>
>>>> [1] https://cwiki.apache.org/confluence/x/J5eqCw
>>>> [2] https://cwiki.apache.org/confluence/x/IJeqCw
>>>>
>>>> Cheers,
>>>> Till
>>>>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Ingo Bürk <in...@ververica.com>.
Hi Till,
from my (admittedly limited) experience with how far projects lag behind in
terms of Flink versions – yes, the combined time it would take to mature
then seems reasonable enough for a sufficient adoption, IMO.
Another reason why I think two releases as a default for the last step
makes sense: say you mature an API to PublicEvolving. Typically, there will
be issues found afterwards. Even if you address these in the very next
release cycle, a duration of one release would mean you fully mature the
API in the same release in which things are still being fixed; intuitively,
it makes sense to me that the step to Public would come after a period of
no changes needed, however.
Ingo
On Fri, Dec 3, 2021 at 4:55 PM Till Rohrmann <tr...@apache.org> wrote:
> Hi Ingo, thanks for your feedback.
>
> Do you think that two release cycles per graduation step would be long
> enough or should it be longer?
>
> Cheers,
> Till
>
> On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
>
> > Hi Till,
> >
> > Overall I whole-heartedly agree with the proposals in this FLIP. Thank
> you
> > for starting this discussion as well! This seems like something that
> could
> > be tested quite nicely with ArchUnit as well; I'll be happy to help
> should
> > the FLIP be accepted.
> >
> > > I would propose per default a single release.
> >
> > The step from PublicEvolving to Public feels more important to me, and I
> > would personally suggest making this transition a bit longer. We have a
> bit
> > of a chicken-egg problem here, because the goal of your FLIP is,
> > ultimately, also to motivate faster adoption of new Flink versions, but
> the
> > status quo prevents that; if we mature APIs too quickly, we risk losing
> out
> > on important feedback. Therefore, I would propose starting slower here,
> and
> > rather think about shortening that cycle in the future.
> >
> >
> > Best
> > Ingo
> >
> > On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> > > Hi everyone,
> > >
> > > As promised, here is the follow-up FLIP [1] for discussing how we can
> > > ensure that newly introduced APIs are being stabilized over time. This
> > FLIP
> > > is related to FLIP-196 [2].
> > >
> > > The idea of FLIP-197 is to introduce an API graduation process that
> > forces
> > > us to increase the API stability guarantee unless there is a very good
> > > reason not to do so. So the proposal is to reverse the process from
> > opt-in
> > > (increasing the stability guarantee explicitly) to opt-out (deciding
> that
> > > an API cannot be graduated with a good reason).
> > >
> > > Since every process breaks if it is not automated, we propose a richer
> > set
> > > of API stability annotations that can capture enough information so
> that
> > we
> > > can implement a test that fails if we fail to follow the process.
> > >
> > > Looking forward to your feedback.
> > >
> > > Hopefully, we can provide our users a better experience when working
> with
> > > Flink because we offer more stable APIs and make them available faster.
> > >
> > > [1] https://cwiki.apache.org/confluence/x/J5eqCw
> > > [2] https://cwiki.apache.org/confluence/x/IJeqCw
> > >
> > > Cheers,
> > > Till
> > >
> >
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Till Rohrmann <tr...@apache.org>.
Hi Ingo, thanks for your feedback.
Do you think that two release cycles per graduation step would be long
enough or should it be longer?
Cheers,
Till
On Fri, Dec 3, 2021 at 4:29 PM Ingo Bürk <in...@ververica.com> wrote:
> Hi Till,
>
> Overall I whole-heartedly agree with the proposals in this FLIP. Thank you
> for starting this discussion as well! This seems like something that could
> be tested quite nicely with ArchUnit as well; I'll be happy to help should
> the FLIP be accepted.
>
> > I would propose per default a single release.
>
> The step from PublicEvolving to Public feels more important to me, and I
> would personally suggest making this transition a bit longer. We have a bit
> of a chicken-egg problem here, because the goal of your FLIP is,
> ultimately, also to motivate faster adoption of new Flink versions, but the
> status quo prevents that; if we mature APIs too quickly, we risk losing out
> on important feedback. Therefore, I would propose starting slower here, and
> rather think about shortening that cycle in the future.
>
>
> Best
> Ingo
>
> On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org> wrote:
>
> > Hi everyone,
> >
> > As promised, here is the follow-up FLIP [1] for discussing how we can
> > ensure that newly introduced APIs are being stabilized over time. This
> FLIP
> > is related to FLIP-196 [2].
> >
> > The idea of FLIP-197 is to introduce an API graduation process that
> forces
> > us to increase the API stability guarantee unless there is a very good
> > reason not to do so. So the proposal is to reverse the process from
> opt-in
> > (increasing the stability guarantee explicitly) to opt-out (deciding that
> > an API cannot be graduated with a good reason).
> >
> > Since every process breaks if it is not automated, we propose a richer
> set
> > of API stability annotations that can capture enough information so that
> we
> > can implement a test that fails if we fail to follow the process.
> >
> > Looking forward to your feedback.
> >
> > Hopefully, we can provide our users a better experience when working with
> > Flink because we offer more stable APIs and make them available faster.
> >
> > [1] https://cwiki.apache.org/confluence/x/J5eqCw
> > [2] https://cwiki.apache.org/confluence/x/IJeqCw
> >
> > Cheers,
> > Till
> >
>
Re: [DISCUSS] FLIP-197: API stability graduation process
Posted by Ingo Bürk <in...@ververica.com>.
Hi Till,
Overall I whole-heartedly agree with the proposals in this FLIP. Thank you
for starting this discussion as well! This seems like something that could
be tested quite nicely with ArchUnit as well; I'll be happy to help should
the FLIP be accepted.
> I would propose per default a single release.
The step from PublicEvolving to Public feels more important to me, and I
would personally suggest making this transition a bit longer. We have a bit
of a chicken-egg problem here, because the goal of your FLIP is,
ultimately, also to motivate faster adoption of new Flink versions, but the
status quo prevents that; if we mature APIs too quickly, we risk losing out
on important feedback. Therefore, I would propose starting slower here, and
rather think about shortening that cycle in the future.
Best
Ingo
On Thu, Dec 2, 2021 at 3:57 PM Till Rohrmann <tr...@apache.org> wrote:
> Hi everyone,
>
> As promised, here is the follow-up FLIP [1] for discussing how we can
> ensure that newly introduced APIs are being stabilized over time. This FLIP
> is related to FLIP-196 [2].
>
> The idea of FLIP-197 is to introduce an API graduation process that forces
> us to increase the API stability guarantee unless there is a very good
> reason not to do so. So the proposal is to reverse the process from opt-in
> (increasing the stability guarantee explicitly) to opt-out (deciding that
> an API cannot be graduated with a good reason).
>
> Since every process breaks if it is not automated, we propose a richer set
> of API stability annotations that can capture enough information so that we
> can implement a test that fails if we fail to follow the process.
>
> Looking forward to your feedback.
>
> Hopefully, we can provide our users a better experience when working with
> Flink because we offer more stable APIs and make them available faster.
>
> [1] https://cwiki.apache.org/confluence/x/J5eqCw
> [2] https://cwiki.apache.org/confluence/x/IJeqCw
>
> Cheers,
> Till
>