Re: [DISCUSS] Releasing Flink 1.14.1

[Qingsheng Ren <re...@gmail.com>]

Hi all,

We found a bug in KafkaSource and consider as a blocker of releasing 1.14.1:

https://issues.apache.org/jira/browse/FLINK-25132 - KafkaSource cannot work with object-reusing DeserializationSchema

This will affect using raw format in Kafka table source, which is a quite common use case.

I’ve submitted a PR and it’s in review now. Hopefully can be merge in a day or two.

--
Best Regards,

Qingsheng Ren
Email: renqschn@gmail.com
On Nov 29, 2021, 5:16 PM +0800, Martijn Visser <ma...@ververica.com>, wrote:
> Thanks all for the updates! To summarize, these are open tickets that are
> considered blockers for Flink 1.14.1:
>
> * https://issues.apache.org/jira/browse/FLINK-22113 - UniqueKey constraint
> is lost with multiple sources join in SQL -> @Marios Trivyzas
> <ma...@gmail.com> can you give an estimate when you expect this to be
> resolved?
> * https://issues.apache.org/jira/browse/FLINK-23946 - Application mode
> fails fatally when being shut down -> A patch is being prepared. @David
> Morávek <dm...@apache.org> do you have an estimate when this patch will be
> there?
> * https://issues.apache.org/jira/browse/FLINK-24596 - Bugs in
> sink.buffer-flush before upsert-kafka -> @fpaul@apache.org
> <fp...@apache.org> has provided a PR is there, so I suspect it would take a
> couple of days before this is merged.
> * https://issues.apache.org/jira/browse/FLINK-25022 - ClassLoader leak with
> ThreadLocals on the JM when submitting a job through the REST API -> @Chesnay
> Schepler <ch...@apache.org> has provided a PR, so I suspect it would also
> just take a couple of days before this is merged.
>
> Is there anyone who can help me with creating the actual release when these
> tickets are resolved
>
> Best regards,
>
> Martijn
>
>
> On Fri, 26 Nov 2021 at 12:08, Chesnay Schepler <ch...@apache.org> wrote:
>
> > FLINK-25022: I will open a PR later today, and it should be easy to
> > backport.
> > FLINK-25027: Unlikely to make it for 1.14.1; I also wouldn't consider it
> > a blocker
> >
> > On 24/11/2021 19:40, Martijn Visser wrote:
> > > Hi all,
> > >
> > > I would like to start a discussion on releasing Flink 1.14.1. Flink 1.14
> > > was released on the 29th of September [1] and so far 107 issues have been
> > > resolved, including multiple blockers and critical priorities [2].
> > >
> > > There are currently 169 open tickets which contain a fixVersion for
> > 1.14.1
> > > [3]. I'm including the ones that are currently marked as critical or a
> > > blocker to verify if these should be included in Flink 1.14.1. It would
> > be
> > > great if those that are assigned or working on one or more of these
> > tickets
> > > can give an update on its status.
> > >
> > > * https://issues.apache.org/jira/browse/FLINK-24543 - Zookeeper
> > connection
> > > issue causes inconsistent state in Flink -> I think this depends on the
> > > outcome of dropping Zookeeper 3.4 as was proposed on the Dev mailing list
> > > * https://issues.apache.org/jira/browse/FLINK-25027 - Allow GC of a
> > > finished job's JobMaster before the slot timeout is reached
> > > * https://issues.apache.org/jira/browse/FLINK-25022 - ClassLoader leak
> > with
> > > ThreadLocals on the JM when submitting a job through the REST API
> > > * https://issues.apache.org/jira/browse/FLINK-24789 -
> > IllegalStateException
> > > with CheckpointCleaner being closed already
> > > * https://issues.apache.org/jira/browse/FLINK-24328 - Long term fix for
> > > receiving new buffer size before network reader configured -> I'm not
> > sure
> > > if this would end up in Flink 1.14.1, I think it's more likely that it
> > > would be Flink 1.15. Anton/Dawid, could you confirm this?
> > > * https://issues.apache.org/jira/browse/FLINK-23946 - Application mode
> > > fails fatally when being shut down -> This depends on
> > > https://issues.apache.org/jira/browse/FLINK-24038 and I don't see much
> > > happening there, so I also expect that this would move to Flink 1.15.
> > > David, could you confirm?
> > > * https://issues.apache.org/jira/browse/FLINK-22113 - UniqueKey
> > constraint
> > > is lost with multiple sources join in SQL
> > > * https://issues.apache.org/jira/browse/FLINK-21788 - Throw
> > > PartitionNotFoundException if the partition file has been lost for
> > blocking
> > > shuffle -> I'm also expecting that this would move to Flink 1.15, can you
> > > confirm Yingjie ?
> > >
> > > There are quite some other tickets that I've excluded from this list,
> > > because they are either test instabilities or are not depending on a
> > Flink
> > > release to be resolved.
> > >
> > > Note: there are quite a few test instabilities in the list and help on
> > > those is always appreciated. You can check all unassigned tickets
> > > instabilities in Jira [4].
> > >
> > > Are there any other open tickets that we should wait for? Is there a PMC
> > > member who would like to manage the release? I'm more than happy to help
> > > with monitoring the status of the tickets.
> > >
> > > Best regards,
> > >
> > > Martijn
> > >
> > > [1] https://flink.apache.org/news/2021/09/29/release-1.14.0.html
> > > [2]
> > >
> > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Resolved%2C%20Closed)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC
> > > [3]
> > >
> > https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC
> > >
> > > [4]
> > >
> > https://issues.apache.org/jira/issues?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.14.1%20AND%20labels%20%3D%20test-stability%20AND%20assignee%20in%20(EMPTY)%20ORDER%20BY%20priority%20DESC%2C%20created%20DESC
> > >
> > > Martijn Visser | Product Manager
> > >
> > > martijn@ververica.com
> > >
> > > <https://www.ververica.com/>
> > >
> > >
> > > Follow us @VervericaData
> > >
> > > --
> > >
> > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> > > Conference
> > >
> > > Stream Processing | Event Driven | Real Time
> > >
> >
> >