You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Yu Li <ca...@gmail.com> on 2022/09/01 01:49:24 UTC
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
*bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x
(or intermediate releases)*
Yes, this might help users to be more prepared before upgrading, if they
could know whether need to recompile their applications. Asking about
1.14/1.15 since they are the "in service" versions.
But it's totally fine if we're targeting for the future (smile).
Best Regards,
Yu
On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler <ch...@apache.org> wrote:
> Well then someone didnt update the FLIP number as they should!
> I'll increment the number of this FLIP to 255.
>
> On 31/08/2022 15:06, Matthias Pohl wrote:
> > +1 for bringing this into a consistent state. Thanks, Chesnay.
> >
> > nit: There's a conflict between this FLIP-254 and the FLIP-254 on the
> redis
> > streams connector.
> >
> > On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler <ch...@apache.org>
> wrote:
> >
> >> * backport to 1.14/1.15
> >>
> >> On 31/08/2022 14:45, Chesnay Schepler wrote:
> >>> @Yu I haven't really considered 1.14/1.15.
> >>>
> >>> What exactly are you interested in; a list of all breaking changes
> >>> between 1.15.0 and the latest 1.15.x (or intermediate releases),
> >>> or are you suggesting to also backport this whole thing to 1.16 (which
> >>> should be possible)?
> >>>
> >>> On 31/08/2022 13:31, Yu Li wrote:
> >>>> +1 for the FLIP. Thanks for the efforts Chesnay.
> >>>>
> >>>> I believe ensuring binary compatibility for patch releases will also
> >>>> benefit our end users besides the cloud service providers.
> >>>>
> >>>> I'm also wondering about the compatibility checking result after
> >>>> enabling
> >>>> japicmp for all modules with existing patch releases (1.14.x and
> >>>> 1.15.x).
> >>>> Do you already have one with your local customized japicmp or do we
> >>>> need to
> >>>> wait until the works tracked in JIRA are done? (smile)
> >>>>
> >>>> Best Regards,
> >>>> Yu
> >>>>
> >>>>
> >>>> On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf <kn...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hi Chesnay,
> >>>>>
> >>>>> thanks for bringing this up and for your research and fixes to
> japicmd.
> >>>>>
> >>>>> +1 for the proposal. For Immerok as an Apache Flink cloud service
> >>>>> provider
> >>>>> it is very valuable to know that our users don't need to upgrade
> >>>>> their Jobs
> >>>>> when the Flink patch version changes. I am sure the same is true for
> >>>>> internal platform teams as well as end users of Apache Flink.
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Konstantin
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler <
> >>>>> chesnay@apache.org>:
> >>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I just published a FLIP to guarantee binary compatibility for patch
> >>>>>> releases. I don't think our current guarantees of
> source-compatibility
> >>>>>> are sufficient for patch releases.
> >>>>>>
> >>>>>>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857
> >>>>>> Let me know what you think.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Chesnay
> >>>>>>
> >>>>> --
> >>>>> https://twitter.com/snntrable
> >>>>> https://github.com/knaufk
> >>>>>
> >>
>
>
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Posted by Yu Li <ca...@gmail.com>.
Thanks for the follow-up Chesnay, I believe this compatible result is
useful, and +1 for applying the FLIP to 1.15.
And yes, from the result we are doing a good job keeping the binary
compatibility in the 1.15 branch :-)
Best Regards,
Yu
On Thu, 1 Sept 2022 at 19:54, Chesnay Schepler <ch...@apache.org> wrote:
> I compared 1.15.0 and 1.15.2; we overall did a very good job with one
> outlier.
> Based on this I'd say we should also apply this FLIP to 1.15.
>
> Table connectors:
> Source-compatible change; relevant if you implement your own
> DataStream(Scan/Sink)Provider:
>
> org.apache.flink.table.connector.sink.DataStreamSinkProvider.consumeDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.datastream.DataStream):METHOD_ABSTRACT_NOW_DEFAULT
>
> org.apache.flink.table.connector.source.DataStreamScanProvider.produceDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.environment.StreamExecutionEnvironment):METHOD_ABSTRACT_NOW_DEFAULT
>
> Pulsar connector:
> These are due to adding @Internal in 1.15.2; no effect on compatibility:
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.getMessageId():METHOD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.valueOf(java.lang.String):METHOD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.values():METHOD_REMOVED,java.lang.Comparable[java.lang.Comparable]:INTERFACE_REMOVED,java.io.Serializable[
> java.io.Serializable]:INTERFACE_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:SUPERCLASS_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.TIMESTAMP:FIELD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.MESSAGE_ID:FIELD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:CLASS_REMOVED
> These are straight-up breaking changes (neither source nor binary
> compatible):
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.seekPosition(org.apache.pulsar.client.api.Consumer):METHOD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.fromPublishTime(long):METHOD_NEW_DEFAULT
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.seekPosition(java.lang.String,int,org.apache.pulsar.client.api.Consumer):METHOD_REMOVED
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterEventTime(long):METHOD_NEW_DEFAULT,org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterPublishTime(long):METHOD_NEW_DEFAULT
>
> org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.shouldStop(org.apache.pulsar.client.api.Message):METHOD_RETURN_TYPE_CHANGED
>
>
> On 01/09/2022 03:49, Yu Li wrote:
> > *bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x
> > (or intermediate releases)*
> > Yes, this might help users to be more prepared before upgrading, if they
> > could know whether need to recompile their applications. Asking about
> > 1.14/1.15 since they are the "in service" versions.
> >
> > But it's totally fine if we're targeting for the future (smile).
> >
> > Best Regards,
> > Yu
> >
> >
> > On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler <ch...@apache.org>
> wrote:
> >
> >> Well then someone didnt update the FLIP number as they should!
> >> I'll increment the number of this FLIP to 255.
> >>
> >> On 31/08/2022 15:06, Matthias Pohl wrote:
> >>> +1 for bringing this into a consistent state. Thanks, Chesnay.
> >>>
> >>> nit: There's a conflict between this FLIP-254 and the FLIP-254 on the
> >> redis
> >>> streams connector.
> >>>
> >>> On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler <ch...@apache.org>
> >> wrote:
> >>>> * backport to 1.14/1.15
> >>>>
> >>>> On 31/08/2022 14:45, Chesnay Schepler wrote:
> >>>>> @Yu I haven't really considered 1.14/1.15.
> >>>>>
> >>>>> What exactly are you interested in; a list of all breaking changes
> >>>>> between 1.15.0 and the latest 1.15.x (or intermediate releases),
> >>>>> or are you suggesting to also backport this whole thing to 1.16
> (which
> >>>>> should be possible)?
> >>>>>
> >>>>> On 31/08/2022 13:31, Yu Li wrote:
> >>>>>> +1 for the FLIP. Thanks for the efforts Chesnay.
> >>>>>>
> >>>>>> I believe ensuring binary compatibility for patch releases will also
> >>>>>> benefit our end users besides the cloud service providers.
> >>>>>>
> >>>>>> I'm also wondering about the compatibility checking result after
> >>>>>> enabling
> >>>>>> japicmp for all modules with existing patch releases (1.14.x and
> >>>>>> 1.15.x).
> >>>>>> Do you already have one with your local customized japicmp or do we
> >>>>>> need to
> >>>>>> wait until the works tracked in JIRA are done? (smile)
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Yu
> >>>>>>
> >>>>>>
> >>>>>> On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf <kn...@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Chesnay,
> >>>>>>>
> >>>>>>> thanks for bringing this up and for your research and fixes to
> >> japicmd.
> >>>>>>> +1 for the proposal. For Immerok as an Apache Flink cloud service
> >>>>>>> provider
> >>>>>>> it is very valuable to know that our users don't need to upgrade
> >>>>>>> their Jobs
> >>>>>>> when the Flink patch version changes. I am sure the same is true
> for
> >>>>>>> internal platform teams as well as end users of Apache Flink.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Konstantin
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler <
> >>>>>>> chesnay@apache.org>:
> >>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> I just published a FLIP to guarantee binary compatibility for
> patch
> >>>>>>>> releases. I don't think our current guarantees of
> >> source-compatibility
> >>>>>>>> are sufficient for patch releases.
> >>>>>>>>
> >>>>>>>>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857
> >>>>>>>> Let me know what you think.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Chesnay
> >>>>>>>>
> >>>>>>> --
> >>>>>>> https://twitter.com/snntrable
> >>>>>>> https://github.com/knaufk
> >>>>>>>
> >>
>
>
Re: [DISCUSS] FLIP-254 Guarantee binary compatibility for Public/-Evolving APIs between patch releases
Posted by Chesnay Schepler <ch...@apache.org>.
I compared 1.15.0 and 1.15.2; we overall did a very good job with one
outlier.
Based on this I'd say we should also apply this FLIP to 1.15.
Table connectors:
Source-compatible change; relevant if you implement your own
DataStream(Scan/Sink)Provider:
org.apache.flink.table.connector.sink.DataStreamSinkProvider.consumeDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.datastream.DataStream):METHOD_ABSTRACT_NOW_DEFAULT
org.apache.flink.table.connector.source.DataStreamScanProvider.produceDataStream(org.apache.flink.table.connector.ProviderContext,org.apache.flink.streaming.api.environment.StreamExecutionEnvironment):METHOD_ABSTRACT_NOW_DEFAULT
Pulsar connector:
These are due to adding @Internal in 1.15.2; no effect on compatibility:
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.getMessageId():METHOD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.valueOf(java.lang.String):METHOD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.values():METHOD_REMOVED,java.lang.Comparable[java.lang.Comparable]:INTERFACE_REMOVED,java.io.Serializable[java.io.Serializable]:INTERFACE_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:SUPERCLASS_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.TIMESTAMP:FIELD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type.MESSAGE_ID:FIELD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition$Type:CLASS_REMOVED
These are straight-up breaking changes (neither source nor binary
compatible):
org.apache.flink.connector.pulsar.source.enumerator.cursor.CursorPosition.seekPosition(org.apache.pulsar.client.api.Consumer):METHOD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.fromPublishTime(long):METHOD_NEW_DEFAULT
org.apache.flink.connector.pulsar.source.enumerator.cursor.StartCursor.seekPosition(java.lang.String,int,org.apache.pulsar.client.api.Consumer):METHOD_REMOVED
org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterEventTime(long):METHOD_NEW_DEFAULT,org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.afterPublishTime(long):METHOD_NEW_DEFAULT
org.apache.flink.connector.pulsar.source.enumerator.cursor.StopCursor.shouldStop(org.apache.pulsar.client.api.Message):METHOD_RETURN_TYPE_CHANGED
On 01/09/2022 03:49, Yu Li wrote:
> *bq. a list of all breaking changes between 1.15.0 and the latest 1.15.x
> (or intermediate releases)*
> Yes, this might help users to be more prepared before upgrading, if they
> could know whether need to recompile their applications. Asking about
> 1.14/1.15 since they are the "in service" versions.
>
> But it's totally fine if we're targeting for the future (smile).
>
> Best Regards,
> Yu
>
>
> On Wed, 31 Aug 2022 at 21:41, Chesnay Schepler <ch...@apache.org> wrote:
>
>> Well then someone didnt update the FLIP number as they should!
>> I'll increment the number of this FLIP to 255.
>>
>> On 31/08/2022 15:06, Matthias Pohl wrote:
>>> +1 for bringing this into a consistent state. Thanks, Chesnay.
>>>
>>> nit: There's a conflict between this FLIP-254 and the FLIP-254 on the
>> redis
>>> streams connector.
>>>
>>> On Wed, Aug 31, 2022 at 2:52 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>>> * backport to 1.14/1.15
>>>>
>>>> On 31/08/2022 14:45, Chesnay Schepler wrote:
>>>>> @Yu I haven't really considered 1.14/1.15.
>>>>>
>>>>> What exactly are you interested in; a list of all breaking changes
>>>>> between 1.15.0 and the latest 1.15.x (or intermediate releases),
>>>>> or are you suggesting to also backport this whole thing to 1.16 (which
>>>>> should be possible)?
>>>>>
>>>>> On 31/08/2022 13:31, Yu Li wrote:
>>>>>> +1 for the FLIP. Thanks for the efforts Chesnay.
>>>>>>
>>>>>> I believe ensuring binary compatibility for patch releases will also
>>>>>> benefit our end users besides the cloud service providers.
>>>>>>
>>>>>> I'm also wondering about the compatibility checking result after
>>>>>> enabling
>>>>>> japicmp for all modules with existing patch releases (1.14.x and
>>>>>> 1.15.x).
>>>>>> Do you already have one with your local customized japicmp or do we
>>>>>> need to
>>>>>> wait until the works tracked in JIRA are done? (smile)
>>>>>>
>>>>>> Best Regards,
>>>>>> Yu
>>>>>>
>>>>>>
>>>>>> On Wed, 31 Aug 2022 at 18:50, Konstantin Knauf <kn...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Chesnay,
>>>>>>>
>>>>>>> thanks for bringing this up and for your research and fixes to
>> japicmd.
>>>>>>> +1 for the proposal. For Immerok as an Apache Flink cloud service
>>>>>>> provider
>>>>>>> it is very valuable to know that our users don't need to upgrade
>>>>>>> their Jobs
>>>>>>> when the Flink patch version changes. I am sure the same is true for
>>>>>>> internal platform teams as well as end users of Apache Flink.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Konstantin
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Am Mi., 31. Aug. 2022 um 12:31 Uhr schrieb Chesnay Schepler <
>>>>>>> chesnay@apache.org>:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I just published a FLIP to guarantee binary compatibility for patch
>>>>>>>> releases. I don't think our current guarantees of
>> source-compatibility
>>>>>>>> are sufficient for patch releases.
>>>>>>>>
>>>>>>>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=225152857
>>>>>>>> Let me know what you think.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Chesnay
>>>>>>>>
>>>>>>> --
>>>>>>> https://twitter.com/snntrable
>>>>>>> https://github.com/knaufk
>>>>>>>
>>