You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Zhu Zhu <re...@gmail.com> on 2020/09/02 12:19:12 UTC
[DISCUSS] Releasing Flink 1.11.2
Hi All,
It has been about 1 month since we released Flink 1.11.1. It's not too far
but
we already have more than 80 resolved improvements/bugs in the release-1.11
branch. Some of them are quite critical. Therefore, I propose to create the
next
bugfix release 1.11.2 for Flink 1.11.
Most noticeable fixes are:
- FLINK-18769 MiniBatch doesn't work with FLIP-95 source
- FLINK-18902 Cannot serve results of asynchronous REST operations in
per-job mode
- FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
- FLINK-18608 CustomizedConvertRule#convertCast drops nullability
- FLINK-18646 Managed memory released check can block RPC thread
- FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
JobManagerFlinkMemoryUtils.java
- FLINK-18663 RestServerEndpoint may prevent server shutdown
- FLINK-18595 Deadlock during job shutdown
- FLINK-18581 Cannot find GC cleaner with java version previous
jdk8u72(-b01)
- FLINK-17075 Add task status reconciliation between TM and JM
Furthermore, I think the following blocker issue should be merged before
1.11.2 release
- FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
I would volunteer as the release manager and kick off the release process.
What do you think?
Please let me know if there are any concerns or any missing blocker issues
need to be fixed in 1.11.2.
Thanks,
Zhu Zhu
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
Hi All,
Just an update.
All known blockers are resolved and I'm starting to create RC1 for release
1.11.2.
Thanks,
Zhu
Zhu Zhu <re...@gmail.com> 于2020年9月9日周三 上午11:36写道：
> Thanks for reporting this issue and offering to fix it @Jingsong Li
> <ji...@gmail.com>
> Agreed it is a reasonable blocker. I will postpone 1.11.2 RC1 creation
> until it is fixed.
>
> Thanks,
> Zhu
>
> Jingsong Li <ji...@gmail.com> 于2020年9月9日周三 上午11:27写道：
>
>> Hi Zhu Zhu,
>>
>> Replenish its[1] influence:
>> For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
>> partition visible for downstream Hive/Spark engines.
>> But due to FLINK-19166, will lose some partitions to commit after Job
>> failover in some cases, especially for short partitions.
>> In the user's opinion, the data is lost, which does not conform to
>> exactly-once.
>>
>> [1]https://issues.apache.org/jira/browse/FLINK-19166
>> [2]
>>
>> https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit
>>
>> Best,
>> Jingsong
>>
>> On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <ji...@gmail.com>
>> wrote:
>>
>> > Hi Zhu Zhu,
>> >
>> > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
>> >
>> > Will fix it soon.
>> >
>> > Best,
>> > Jingsong
>> >
>> > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <re...@gmail.com> wrote:
>> >
>> >> Hi All,
>> >>
>> >> Since there are still two 1.11.2 blockers items in progress, RC1
>> creation
>> >> will be postponed to tomorrow.
>> >>
>> >> Thanks,
>> >> Zhu
>> >>
>> >> Zhu Zhu <re...@gmail.com> 于2020年9月4日周五 下午3:50写道：
>> >>
>> >> > @Dawid
>> >> > Thanks for the information and preparing a fix for FLINK-19133!
>> >> > I have made it a blocker for 1.11.2 and will keep tracking its
>> status.
>> >> >
>> >> > @Till
>> >> > Thanks for the updates and efforts for FLINK-18959!
>> >> >
>> >> > Thanks,
>> >> > Zhu
>> >> >
>> >> > Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
>> >> >
>> >> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>> >> >>
>> >> >> Cheers,
>> >> >> Till
>> >> >>
>> >> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
>> >> dwysakowicz@apache.org>
>> >> >> wrote:
>> >> >>
>> >> >>> User has just reported another issue FLINK-19133 which I think
>> should
>> >> be
>> >> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as
>> soon as
>> >> >>> possible.
>> >> >>>
>> >> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> >> >>> > Thanks for the inputs!
>> >> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >> >>> >
>> >> >>> > And thanks a lot for offering help, zhijiang!
>> >> >>> >
>> >> >>> > Thanks,
>> >> >>> > Zhu
>> >> >>> >
>> >> >>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
>> >> >>> >
>> >> >>> >> Hi
>> >> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942
>> (this
>> >> >>> fixes a
>> >> >>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >> >>> >> Best,
>> >> >>> >> Congxian
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>> >> >>> >>
>> >> >>> >>> Hi,
>> >> >>> >>>
>> >> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
>> >> blocker
>> >> >>> for
>> >> >>> >>> Pravega connector integration, hope we can have it included in
>> >> 1.11.2
>> >> >>> >>> release.
>> >> >>> >>>
>> >> >>> >>> Best Regards,
>> >> >>> >>> Brian
>> >> >>> >>>
>> >> >>> >>> -----Original Message-----
>> >> >>> >>> From: Becket Qin <be...@gmail.com>
>> >> >>> >>> Sent: Thursday, September 3, 2020 11:18
>> >> >>> >>> To: dev
>> >> >>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david;
>> Jingsong
>> >> Li
>> >> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >> >>> >>>
>> >> >>> >>>
>> >> >>> >>> [EXTERNAL EMAIL]
>> >> >>> >>>
>> >> >>> >>> Hi Zhuzhu,
>> >> >>> >>>
>> >> >>> >>> Thanks for starting the discussion.
>> >> >>> >>>
>> >> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> >> >>> regression
>> >> >>> >>> from previous versions and is currently blocking the
>> development
>> >> of
>> >> >>> >> Pravega
>> >> >>> >>> source on top of FLIP-27.
>> >> >>> >>>
>> >> >>> >>> Thanks,
>> >> >>> >>>
>> >> >>> >>> Jiangjie (Becket) Qin
>> >> >>> >>>
>> >> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com>
>> >> wrote:
>> >> >>> >>>
>> >> >>> >>>> Thank you all for the inputs!
>> >> >>> >>>>
>> >> >>> >>>> I agree with Till that we should set a soft deadline first.
>> >> >>> >>>> I'd like to propose next Monday if no new blocker issue pops
>> up.
>> >> >>> >>>> But feel free to raise your concerns if you feel next Monday
>> as a
>> >> >>> >>>> deadline may not work for fixes which should be a blocker for
>> >> >>> 1.11.2.
>> >> >>> >>>>
>> >> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >> >>> HiveBulkWriterFactory
>> >> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
>> >> archiveExecutionGraph
>> >> >>> >>>> because job is not finished when dispatcher close
>> >> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in
>> connected
>> >> >>> >>>> stream
>> >> >>> >>>>
>> >> >>> >>>> Thanks,
>> >> >>> >>>> Zhu
>> >> >>> >>>>
>> >> >>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>> >> >>> >>>>
>> >> >>> >>>>> I think it would be nice to include a fix for
>> >> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as
>> it
>> >> >>> >>>>> affects a highly requested feature of Flink 1.11 quite
>> severely.
>> >> >>> >>>>>
>> >> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
>> >> trohrmann@apache.org
>> >> >>> >
>> >> >>> >>>> wrote:
>> >> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >> >>> >>>>>> volunteering
>> >> >>> >>>> as
>> >> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug
>> fix
>> >> >>> >>>> release. I
>> >> >>> >>>>>> think we already collected quite a bit of fixes which our
>> users
>> >> >>> >>>>>> will benefit from.
>> >> >>> >>>>>>
>> >> >>> >>>>>> For the pending fixes, I would suggest setting a soft
>> deadline
>> >> >>> >>>>>> (maybe
>> >> >>> >>>>> until
>> >> >>> >>>>>> beginning of next week) and then starting to cut the release
>> >> >>> >>>>>> (given
>> >> >>> >>>> that
>> >> >>> >>>>> no
>> >> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve
>> the
>> >> >>> >>>>>> issues
>> >> >>> >>>>> even
>> >> >>> >>>>>> earlier which would allow us to cut the release also
>> earlier.
>> >> >>> >>>>>>
>> >> >>> >>>>>> From my side I would like to include FLINK-18959 in the
>> >> release.
>> >> >>> >>>>>> But it
>> >> >>> >>>>> is
>> >> >>> >>>>>> not a strict release blocker.
>> >> >>> >>>>>>
>> >> >>> >>>>>> Cheers,
>> >> >>> >>>>>> Till
>> >> >>> >>>>>>
>> >> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >> >>> >>>>>> <da...@alpinegizmo.com>
>> >> >>> >>>>>> wrote:
>> >> >>> >>>>>>
>> >> >>> >>>>>>> I think it's worth considering whether we can get this
>> bugfix
>> >> >>> >>>> included
>> >> >>> >>>>> in
>> >> >>> >>>>>>> 1.11.2:
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
>> >> Roman,
>> >> >>> >>>>>>> who
>> >> >>> >>>> has
>> >> >>> >>>>>>> been working on this.
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> Regards,
>> >> >>> >>>>>>> David
>> >> >>> >>>>>>>
>> >> >>> >>>>>>>
>> >> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
>> >> >>> >> wrote:
>> >> >>> >>>>>>>> Hi All,
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
>> >> It's
>> >> >>> >>>>>>>> not
>> >> >>> >>>>> too
>> >> >>> >>>>>>> far
>> >> >>> >>>>>>>> but
>> >> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
>> >> the
>> >> >>> >>>>>>> release-1.11
>> >> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I
>> propose
>> >> >>> >>>>>>>> to
>> >> >>> >>>>> create
>> >> >>> >>>>>>> the
>> >> >>> >>>>>>>> next
>> >> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Most noticeable fixes are:
>> >> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >> >>> >>>>>>>> operations
>> >> >>> >>>> in
>> >> >>> >>>>>>>> per-job mode
>> >> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0
>> table
>> >> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >> >>> >>>>>>>> nullability
>> >> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >> >>> >>>>>>>> thread
>> >> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >> >>> >>>>>>>> incorrectly
>> >> >>> >>>>> in
>> >> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server
>> shutdown
>> >> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >> >>> >>>>>>>> previous
>> >> >>> >>>>>>>> jdk8u72(-b01)
>> >> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM
>> and
>> >> JM
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >> >>> >>>>>>>> merged
>> >> >>> >>>>>> before
>> >> >>> >>>>>>>> 1.11.2 release
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >> >>> >>>>> HiveBulkWriterFactory
>> >> >>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >> >>> >>>>>>>> release
>> >> >>> >>>>>>> process.
>> >> >>> >>>>>>>> What do you think?
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Please let me know if there are any concerns or any
>> missing
>> >> >>> >>>>>>>> blocker
>> >> >>> >>>>>>> issues
>> >> >>> >>>>>>>> need to be fixed in 1.11.2.
>> >> >>> >>>>>>>>
>> >> >>> >>>>>>>> Thanks,
>> >> >>> >>>>>>>> Zhu Zhu
>> >> >>> >>>>>>>>
>> >> >>> >>>>>
>> >> >>> >>>>> --
>> >> >>> >>>>>
>> >> >>> >>>>> Konstantin Knauf
>> >> >>> >>>>>
>> >> >>> >>>>> https://twitter.com/snntrable
>> >> >>> >>>>>
>> >> >>> >>>>> https://github.com/knaufk
>> >> >>> >>>>>
>> >> >>>
>> >> >>>
>> >>
>> >
>> >
>> > --
>> > Best, Jingsong Lee
>> >
>>
>>
>> --
>> Best, Jingsong Lee
>>
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
Thanks for reporting this issue and offering to fix it @Jingsong Li
<ji...@gmail.com>
Agreed it is a reasonable blocker. I will postpone 1.11.2 RC1 creation
until it is fixed.
Thanks,
Zhu
Jingsong Li <ji...@gmail.com> 于2020年9月9日周三 上午11:27写道：
> Hi Zhu Zhu,
>
> Replenish its[1] influence:
> For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
> partition visible for downstream Hive/Spark engines.
> But due to FLINK-19166, will lose some partitions to commit after Job
> failover in some cases, especially for short partitions.
> In the user's opinion, the data is lost, which does not conform to
> exactly-once.
>
> [1]https://issues.apache.org/jira/browse/FLINK-19166
> [2]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit
>
> Best,
> Jingsong
>
> On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <ji...@gmail.com>
> wrote:
>
> > Hi Zhu Zhu,
> >
> > Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
> >
> > Will fix it soon.
> >
> > Best,
> > Jingsong
> >
> > On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <re...@gmail.com> wrote:
> >
> >> Hi All,
> >>
> >> Since there are still two 1.11.2 blockers items in progress, RC1
> creation
> >> will be postponed to tomorrow.
> >>
> >> Thanks,
> >> Zhu
> >>
> >> Zhu Zhu <re...@gmail.com> 于2020年9月4日周五 下午3:50写道：
> >>
> >> > @Dawid
> >> > Thanks for the information and preparing a fix for FLINK-19133!
> >> > I have made it a blocker for 1.11.2 and will keep tracking its status.
> >> >
> >> > @Till
> >> > Thanks for the updates and efforts for FLINK-18959!
> >> >
> >> > Thanks,
> >> > Zhu
> >> >
> >> > Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
> >> >
> >> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
> >> >>
> >> >> Cheers,
> >> >> Till
> >> >>
> >> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
> >> dwysakowicz@apache.org>
> >> >> wrote:
> >> >>
> >> >>> User has just reported another issue FLINK-19133 which I think
> should
> >> be
> >> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon
> as
> >> >>> possible.
> >> >>>
> >> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
> >> >>> > Thanks for the inputs!
> >> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >> >>> >
> >> >>> > And thanks a lot for offering help, zhijiang!
> >> >>> >
> >> >>> > Thanks,
> >> >>> > Zhu
> >> >>> >
> >> >>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
> >> >>> >
> >> >>> >> Hi
> >> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942
> (this
> >> >>> fixes a
> >> >>> >> bug introduce in 1.11.0), there is a pr for it already.
> >> >>> >> Best,
> >> >>> >> Congxian
> >> >>> >>
> >> >>> >>
> >> >>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
> >> >>> >>
> >> >>> >>> Hi,
> >> >>> >>>
> >> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
> >> blocker
> >> >>> for
> >> >>> >>> Pravega connector integration, hope we can have it included in
> >> 1.11.2
> >> >>> >>> release.
> >> >>> >>>
> >> >>> >>> Best Regards,
> >> >>> >>> Brian
> >> >>> >>>
> >> >>> >>> -----Original Message-----
> >> >>> >>> From: Becket Qin <be...@gmail.com>
> >> >>> >>> Sent: Thursday, September 3, 2020 11:18
> >> >>> >>> To: dev
> >> >>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong
> >> Li
> >> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >> >>> >>>
> >> >>> >>>
> >> >>> >>> [EXTERNAL EMAIL]
> >> >>> >>>
> >> >>> >>> Hi Zhuzhu,
> >> >>> >>>
> >> >>> >>> Thanks for starting the discussion.
> >> >>> >>>
> >> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
> >> >>> regression
> >> >>> >>> from previous versions and is currently blocking the development
> >> of
> >> >>> >> Pravega
> >> >>> >>> source on top of FLIP-27.
> >> >>> >>>
> >> >>> >>> Thanks,
> >> >>> >>>
> >> >>> >>> Jiangjie (Becket) Qin
> >> >>> >>>
> >> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com>
> >> wrote:
> >> >>> >>>
> >> >>> >>>> Thank you all for the inputs!
> >> >>> >>>>
> >> >>> >>>> I agree with Till that we should set a soft deadline first.
> >> >>> >>>> I'd like to propose next Monday if no new blocker issue pops
> up.
> >> >>> >>>> But feel free to raise your concerns if you feel next Monday
> as a
> >> >>> >>>> deadline may not work for fixes which should be a blocker for
> >> >>> 1.11.2.
> >> >>> >>>>
> >> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >> >>> HiveBulkWriterFactory
> >> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
> >> archiveExecutionGraph
> >> >>> >>>> because job is not finished when dispatcher close
> >> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in
> connected
> >> >>> >>>> stream
> >> >>> >>>>
> >> >>> >>>> Thanks,
> >> >>> >>>> Zhu
> >> >>> >>>>
> >> >>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> >> >>> >>>>
> >> >>> >>>>> I think it would be nice to include a fix for
> >> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >> >>> >>>>> affects a highly requested feature of Flink 1.11 quite
> severely.
> >> >>> >>>>>
> >> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
> >> trohrmann@apache.org
> >> >>> >
> >> >>> >>>> wrote:
> >> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >> >>> >>>>>> volunteering
> >> >>> >>>> as
> >> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug
> fix
> >> >>> >>>> release. I
> >> >>> >>>>>> think we already collected quite a bit of fixes which our
> users
> >> >>> >>>>>> will benefit from.
> >> >>> >>>>>>
> >> >>> >>>>>> For the pending fixes, I would suggest setting a soft
> deadline
> >> >>> >>>>>> (maybe
> >> >>> >>>>> until
> >> >>> >>>>>> beginning of next week) and then starting to cut the release
> >> >>> >>>>>> (given
> >> >>> >>>> that
> >> >>> >>>>> no
> >> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve
> the
> >> >>> >>>>>> issues
> >> >>> >>>>> even
> >> >>> >>>>>> earlier which would allow us to cut the release also earlier.
> >> >>> >>>>>>
> >> >>> >>>>>> From my side I would like to include FLINK-18959 in the
> >> release.
> >> >>> >>>>>> But it
> >> >>> >>>>> is
> >> >>> >>>>>> not a strict release blocker.
> >> >>> >>>>>>
> >> >>> >>>>>> Cheers,
> >> >>> >>>>>> Till
> >> >>> >>>>>>
> >> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >> >>> >>>>>> <da...@alpinegizmo.com>
> >> >>> >>>>>> wrote:
> >> >>> >>>>>>
> >> >>> >>>>>>> I think it's worth considering whether we can get this
> bugfix
> >> >>> >>>> included
> >> >>> >>>>> in
> >> >>> >>>>>>> 1.11.2:
> >> >>> >>>>>>>
> >> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >> >>> >>>>>>>
> >> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
> >> Roman,
> >> >>> >>>>>>> who
> >> >>> >>>> has
> >> >>> >>>>>>> been working on this.
> >> >>> >>>>>>>
> >> >>> >>>>>>> Regards,
> >> >>> >>>>>>> David
> >> >>> >>>>>>>
> >> >>> >>>>>>>
> >> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
> >> >>> >> wrote:
> >> >>> >>>>>>>> Hi All,
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
> >> It's
> >> >>> >>>>>>>> not
> >> >>> >>>>> too
> >> >>> >>>>>>> far
> >> >>> >>>>>>>> but
> >> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
> >> the
> >> >>> >>>>>>> release-1.11
> >> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I
> propose
> >> >>> >>>>>>>> to
> >> >>> >>>>> create
> >> >>> >>>>>>> the
> >> >>> >>>>>>>> next
> >> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Most noticeable fixes are:
> >> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >> >>> >>>>>>>> operations
> >> >>> >>>> in
> >> >>> >>>>>>>> per-job mode
> >> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0
> table
> >> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >> >>> >>>>>>>> nullability
> >> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >> >>> >>>>>>>> thread
> >> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >> >>> >>>>>>>> incorrectly
> >> >>> >>>>> in
> >> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server
> shutdown
> >> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >> >>> >>>>>>>> previous
> >> >>> >>>>>>>> jdk8u72(-b01)
> >> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and
> >> JM
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
> >> >>> >>>>>>>> merged
> >> >>> >>>>>> before
> >> >>> >>>>>>>> 1.11.2 release
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >> >>> >>>>> HiveBulkWriterFactory
> >> >>> >>>>>>>> I would volunteer as the release manager and kick off the
> >> >>> >>>>>>>> release
> >> >>> >>>>>>> process.
> >> >>> >>>>>>>> What do you think?
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Please let me know if there are any concerns or any missing
> >> >>> >>>>>>>> blocker
> >> >>> >>>>>>> issues
> >> >>> >>>>>>>> need to be fixed in 1.11.2.
> >> >>> >>>>>>>>
> >> >>> >>>>>>>> Thanks,
> >> >>> >>>>>>>> Zhu Zhu
> >> >>> >>>>>>>>
> >> >>> >>>>>
> >> >>> >>>>> --
> >> >>> >>>>>
> >> >>> >>>>> Konstantin Knauf
> >> >>> >>>>>
> >> >>> >>>>> https://twitter.com/snntrable
> >> >>> >>>>>
> >> >>> >>>>> https://github.com/knaufk
> >> >>> >>>>>
> >> >>>
> >> >>>
> >>
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
>
> --
> Best, Jingsong Lee
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Jingsong Li <ji...@gmail.com>.
Hi Zhu Zhu,
Replenish its[1] influence:
For HiveStreamingSink & FileSystemSink in Table/SQL, partition commit make
partition visible for downstream Hive/Spark engines.
But due to FLINK-19166, will lose some partitions to commit after Job
failover in some cases, especially for short partitions.
In the user's opinion, the data is lost, which does not conform to
exactly-once.
[1]https://issues.apache.org/jira/browse/FLINK-19166
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html#partition-commit
Best,
Jingsong
On Wed, Sep 9, 2020 at 11:08 AM Jingsong Li <ji...@gmail.com> wrote:
> Hi Zhu Zhu,
>
> Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
>
> Will fix it soon.
>
> Best,
> Jingsong
>
> On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <re...@gmail.com> wrote:
>
>> Hi All,
>>
>> Since there are still two 1.11.2 blockers items in progress, RC1 creation
>> will be postponed to tomorrow.
>>
>> Thanks,
>> Zhu
>>
>> Zhu Zhu <re...@gmail.com> 于2020年9月4日周五 下午3:50写道：
>>
>> > @Dawid
>> > Thanks for the information and preparing a fix for FLINK-19133!
>> > I have made it a blocker for 1.11.2 and will keep tracking its status.
>> >
>> > @Till
>> > Thanks for the updates and efforts for FLINK-18959!
>> >
>> > Thanks,
>> > Zhu
>> >
>> > Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
>> >
>> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>> >>
>> >> Cheers,
>> >> Till
>> >>
>> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <
>> dwysakowicz@apache.org>
>> >> wrote:
>> >>
>> >>> User has just reported another issue FLINK-19133 which I think should
>> be
>> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>> >>> possible.
>> >>>
>> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> >>> > Thanks for the inputs!
>> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >>> >
>> >>> > And thanks a lot for offering help, zhijiang!
>> >>> >
>> >>> > Thanks,
>> >>> > Zhu
>> >>> >
>> >>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
>> >>> >
>> >>> >> Hi
>> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>> >>> fixes a
>> >>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >>> >> Best,
>> >>> >> Congxian
>> >>> >>
>> >>> >>
>> >>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>> >>> >>
>> >>> >>> Hi,
>> >>> >>>
>> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
>> blocker
>> >>> for
>> >>> >>> Pravega connector integration, hope we can have it included in
>> 1.11.2
>> >>> >>> release.
>> >>> >>>
>> >>> >>> Best Regards,
>> >>> >>> Brian
>> >>> >>>
>> >>> >>> -----Original Message-----
>> >>> >>> From: Becket Qin <be...@gmail.com>
>> >>> >>> Sent: Thursday, September 3, 2020 11:18
>> >>> >>> To: dev
>> >>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong
>> Li
>> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >>> >>>
>> >>> >>>
>> >>> >>> [EXTERNAL EMAIL]
>> >>> >>>
>> >>> >>> Hi Zhuzhu,
>> >>> >>>
>> >>> >>> Thanks for starting the discussion.
>> >>> >>>
>> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> >>> regression
>> >>> >>> from previous versions and is currently blocking the development
>> of
>> >>> >> Pravega
>> >>> >>> source on top of FLIP-27.
>> >>> >>>
>> >>> >>> Thanks,
>> >>> >>>
>> >>> >>> Jiangjie (Becket) Qin
>> >>> >>>
>> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com>
>> wrote:
>> >>> >>>
>> >>> >>>> Thank you all for the inputs!
>> >>> >>>>
>> >>> >>>> I agree with Till that we should set a soft deadline first.
>> >>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>> >>> >>>> But feel free to raise your concerns if you feel next Monday as a
>> >>> >>>> deadline may not work for fixes which should be a blocker for
>> >>> 1.11.2.
>> >>> >>>>
>> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>> HiveBulkWriterFactory
>> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to
>> archiveExecutionGraph
>> >>> >>>> because job is not finished when dispatcher close
>> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>> >>> >>>> stream
>> >>> >>>>
>> >>> >>>> Thanks,
>> >>> >>>> Zhu
>> >>> >>>>
>> >>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>> >>> >>>>
>> >>> >>>>> I think it would be nice to include a fix for
>> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>> >>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>> >>> >>>>>
>> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
>> trohrmann@apache.org
>> >>> >
>> >>> >>>> wrote:
>> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >>> >>>>>> volunteering
>> >>> >>>> as
>> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>> >>> >>>> release. I
>> >>> >>>>>> think we already collected quite a bit of fixes which our users
>> >>> >>>>>> will benefit from.
>> >>> >>>>>>
>> >>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>> >>> >>>>>> (maybe
>> >>> >>>>> until
>> >>> >>>>>> beginning of next week) and then starting to cut the release
>> >>> >>>>>> (given
>> >>> >>>> that
>> >>> >>>>> no
>> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>> >>> >>>>>> issues
>> >>> >>>>> even
>> >>> >>>>>> earlier which would allow us to cut the release also earlier.
>> >>> >>>>>>
>> >>> >>>>>> From my side I would like to include FLINK-18959 in the
>> release.
>> >>> >>>>>> But it
>> >>> >>>>> is
>> >>> >>>>>> not a strict release blocker.
>> >>> >>>>>>
>> >>> >>>>>> Cheers,
>> >>> >>>>>> Till
>> >>> >>>>>>
>> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >>> >>>>>> <da...@alpinegizmo.com>
>> >>> >>>>>> wrote:
>> >>> >>>>>>
>> >>> >>>>>>> I think it's worth considering whether we can get this bugfix
>> >>> >>>> included
>> >>> >>>>> in
>> >>> >>>>>>> 1.11.2:
>> >>> >>>>>>>
>> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>> >>>>>>>
>> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing
>> Roman,
>> >>> >>>>>>> who
>> >>> >>>> has
>> >>> >>>>>>> been working on this.
>> >>> >>>>>>>
>> >>> >>>>>>> Regards,
>> >>> >>>>>>> David
>> >>> >>>>>>>
>> >>> >>>>>>>
>> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
>> >>> >> wrote:
>> >>> >>>>>>>> Hi All,
>> >>> >>>>>>>>
>> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1.
>> It's
>> >>> >>>>>>>> not
>> >>> >>>>> too
>> >>> >>>>>>> far
>> >>> >>>>>>>> but
>> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in
>> the
>> >>> >>>>>>> release-1.11
>> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>> >>> >>>>>>>> to
>> >>> >>>>> create
>> >>> >>>>>>> the
>> >>> >>>>>>>> next
>> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >>> >>>>>>>>
>> >>> >>>>>>>> Most noticeable fixes are:
>> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >>> >>>>>>>> operations
>> >>> >>>> in
>> >>> >>>>>>>> per-job mode
>> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >>> >>>>>>>> nullability
>> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >>> >>>>>>>> thread
>> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >>> >>>>>>>> incorrectly
>> >>> >>>>> in
>> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >>> >>>>>>>> previous
>> >>> >>>>>>>> jdk8u72(-b01)
>> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and
>> JM
>> >>> >>>>>>>>
>> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >>> >>>>>>>> merged
>> >>> >>>>>> before
>> >>> >>>>>>>> 1.11.2 release
>> >>> >>>>>>>>
>> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>> >>>>> HiveBulkWriterFactory
>> >>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >>> >>>>>>>> release
>> >>> >>>>>>> process.
>> >>> >>>>>>>> What do you think?
>> >>> >>>>>>>>
>> >>> >>>>>>>> Please let me know if there are any concerns or any missing
>> >>> >>>>>>>> blocker
>> >>> >>>>>>> issues
>> >>> >>>>>>>> need to be fixed in 1.11.2.
>> >>> >>>>>>>>
>> >>> >>>>>>>> Thanks,
>> >>> >>>>>>>> Zhu Zhu
>> >>> >>>>>>>>
>> >>> >>>>>
>> >>> >>>>> --
>> >>> >>>>>
>> >>> >>>>> Konstantin Knauf
>> >>> >>>>>
>> >>> >>>>> https://twitter.com/snntrable
>> >>> >>>>>
>> >>> >>>>> https://github.com/knaufk
>> >>> >>>>>
>> >>>
>> >>>
>>
>
>
> --
> Best, Jingsong Lee
>
--
Best, Jingsong Lee
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Jingsong Li <ji...@gmail.com>.
Hi Zhu Zhu,
Add a new blocker: https://issues.apache.org/jira/browse/FLINK-19166
Will fix it soon.
Best,
Jingsong
On Tue, Sep 8, 2020 at 12:26 AM Zhu Zhu <re...@gmail.com> wrote:
> Hi All,
>
> Since there are still two 1.11.2 blockers items in progress, RC1 creation
> will be postponed to tomorrow.
>
> Thanks,
> Zhu
>
> Zhu Zhu <re...@gmail.com> 于2020年9月4日周五 下午3:50写道：
>
> > @Dawid
> > Thanks for the information and preparing a fix for FLINK-19133!
> > I have made it a blocker for 1.11.2 and will keep tracking its status.
> >
> > @Till
> > Thanks for the updates and efforts for FLINK-18959!
> >
> > Thanks,
> > Zhu
> >
> > Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
> >
> >> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
> >>
> >> Cheers,
> >> Till
> >>
> >> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <dwysakowicz@apache.org
> >
> >> wrote:
> >>
> >>> User has just reported another issue FLINK-19133 which I think should
> be
> >>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
> >>> possible.
> >>>
> >>> On 03/09/2020 09:36, Zhu Zhu wrote:
> >>> > Thanks for the inputs!
> >>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >>> >
> >>> > And thanks a lot for offering help, zhijiang!
> >>> >
> >>> > Thanks,
> >>> > Zhu
> >>> >
> >>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
> >>> >
> >>> >> Hi
> >>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
> >>> fixes a
> >>> >> bug introduce in 1.11.0), there is a pr for it already.
> >>> >> Best,
> >>> >> Congxian
> >>> >>
> >>> >>
> >>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
> >>> >>
> >>> >>> Hi,
> >>> >>>
> >>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a
> blocker
> >>> for
> >>> >>> Pravega connector integration, hope we can have it included in
> 1.11.2
> >>> >>> release.
> >>> >>>
> >>> >>> Best Regards,
> >>> >>> Brian
> >>> >>>
> >>> >>> -----Original Message-----
> >>> >>> From: Becket Qin <be...@gmail.com>
> >>> >>> Sent: Thursday, September 3, 2020 11:18
> >>> >>> To: dev
> >>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
> >>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >>> >>>
> >>> >>>
> >>> >>> [EXTERNAL EMAIL]
> >>> >>>
> >>> >>> Hi Zhuzhu,
> >>> >>>
> >>> >>> Thanks for starting the discussion.
> >>> >>>
> >>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
> >>> regression
> >>> >>> from previous versions and is currently blocking the development of
> >>> >> Pravega
> >>> >>> source on top of FLIP-27.
> >>> >>>
> >>> >>> Thanks,
> >>> >>>
> >>> >>> Jiangjie (Becket) Qin
> >>> >>>
> >>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
> >>> >>>
> >>> >>>> Thank you all for the inputs!
> >>> >>>>
> >>> >>>> I agree with Till that we should set a soft deadline first.
> >>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
> >>> >>>> But feel free to raise your concerns if you feel next Monday as a
> >>> >>>> deadline may not work for fixes which should be a blocker for
> >>> 1.11.2.
> >>> >>>>
> >>> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>> HiveBulkWriterFactory
> >>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> >>> >>>> because job is not finished when dispatcher close
> >>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
> >>> >>>> stream
> >>> >>>>
> >>> >>>> Thanks,
> >>> >>>> Zhu
> >>> >>>>
> >>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> >>> >>>>
> >>> >>>>> I think it would be nice to include a fix for
> >>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
> >>> >>>>>
> >>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <
> trohrmann@apache.org
> >>> >
> >>> >>>> wrote:
> >>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >>> >>>>>> volunteering
> >>> >>>> as
> >>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
> >>> >>>> release. I
> >>> >>>>>> think we already collected quite a bit of fixes which our users
> >>> >>>>>> will benefit from.
> >>> >>>>>>
> >>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
> >>> >>>>>> (maybe
> >>> >>>>> until
> >>> >>>>>> beginning of next week) and then starting to cut the release
> >>> >>>>>> (given
> >>> >>>> that
> >>> >>>>> no
> >>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
> >>> >>>>>> issues
> >>> >>>>> even
> >>> >>>>>> earlier which would allow us to cut the release also earlier.
> >>> >>>>>>
> >>> >>>>>> From my side I would like to include FLINK-18959 in the release.
> >>> >>>>>> But it
> >>> >>>>> is
> >>> >>>>>> not a strict release blocker.
> >>> >>>>>>
> >>> >>>>>> Cheers,
> >>> >>>>>> Till
> >>> >>>>>>
> >>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >>> >>>>>> <da...@alpinegizmo.com>
> >>> >>>>>> wrote:
> >>> >>>>>>
> >>> >>>>>>> I think it's worth considering whether we can get this bugfix
> >>> >>>> included
> >>> >>>>> in
> >>> >>>>>>> 1.11.2:
> >>> >>>>>>>
> >>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>> >>>>>>>
> >>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
> >>> >>>>>>> who
> >>> >>>> has
> >>> >>>>>>> been working on this.
> >>> >>>>>>>
> >>> >>>>>>> Regards,
> >>> >>>>>>> David
> >>> >>>>>>>
> >>> >>>>>>>
> >>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
> >>> >> wrote:
> >>> >>>>>>>> Hi All,
> >>> >>>>>>>>
> >>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
> >>> >>>>>>>> not
> >>> >>>>> too
> >>> >>>>>>> far
> >>> >>>>>>>> but
> >>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
> >>> >>>>>>> release-1.11
> >>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
> >>> >>>>>>>> to
> >>> >>>>> create
> >>> >>>>>>> the
> >>> >>>>>>>> next
> >>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >>> >>>>>>>>
> >>> >>>>>>>> Most noticeable fixes are:
> >>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >>> >>>>>>>> operations
> >>> >>>> in
> >>> >>>>>>>> per-job mode
> >>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> >>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >>> >>>>>>>> nullability
> >>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >>> >>>>>>>> thread
> >>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >>> >>>>>>>> incorrectly
> >>> >>>>> in
> >>> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> >>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >>> >>>>>>>> previous
> >>> >>>>>>>> jdk8u72(-b01)
> >>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
> >>> >>>>>>>>
> >>> >>>>>>>> Furthermore, I think the following blocker issue should be
> >>> >>>>>>>> merged
> >>> >>>>>> before
> >>> >>>>>>>> 1.11.2 release
> >>> >>>>>>>>
> >>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>> >>>>> HiveBulkWriterFactory
> >>> >>>>>>>> I would volunteer as the release manager and kick off the
> >>> >>>>>>>> release
> >>> >>>>>>> process.
> >>> >>>>>>>> What do you think?
> >>> >>>>>>>>
> >>> >>>>>>>> Please let me know if there are any concerns or any missing
> >>> >>>>>>>> blocker
> >>> >>>>>>> issues
> >>> >>>>>>>> need to be fixed in 1.11.2.
> >>> >>>>>>>>
> >>> >>>>>>>> Thanks,
> >>> >>>>>>>> Zhu Zhu
> >>> >>>>>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>>
> >>> >>>>> Konstantin Knauf
> >>> >>>>>
> >>> >>>>> https://twitter.com/snntrable
> >>> >>>>>
> >>> >>>>> https://github.com/knaufk
> >>> >>>>>
> >>>
> >>>
>
--
Best, Jingsong Lee
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
Hi All,
Since there are still two 1.11.2 blockers items in progress, RC1 creation
will be postponed to tomorrow.
Thanks,
Zhu
Zhu Zhu <re...@gmail.com> 于2020年9月4日周五 下午3:50写道：
> @Dawid
> Thanks for the information and preparing a fix for FLINK-19133!
> I have made it a blocker for 1.11.2 and will keep tracking its status.
>
> @Till
> Thanks for the updates and efforts for FLINK-18959!
>
> Thanks,
> Zhu
>
> Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
>
>> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>>
>> Cheers,
>> Till
>>
>> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <dw...@apache.org>
>> wrote:
>>
>>> User has just reported another issue FLINK-19133 which I think should be
>>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>>> possible.
>>>
>>> On 03/09/2020 09:36, Zhu Zhu wrote:
>>> > Thanks for the inputs!
>>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>>> >
>>> > And thanks a lot for offering help, zhijiang!
>>> >
>>> > Thanks,
>>> > Zhu
>>> >
>>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
>>> >
>>> >> Hi
>>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>>> fixes a
>>> >> bug introduce in 1.11.0), there is a pr for it already.
>>> >> Best,
>>> >> Congxian
>>> >>
>>> >>
>>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
>>> for
>>> >>> Pravega connector integration, hope we can have it included in 1.11.2
>>> >>> release.
>>> >>>
>>> >>> Best Regards,
>>> >>> Brian
>>> >>>
>>> >>> -----Original Message-----
>>> >>> From: Becket Qin <be...@gmail.com>
>>> >>> Sent: Thursday, September 3, 2020 11:18
>>> >>> To: dev
>>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
>>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>>> >>>
>>> >>>
>>> >>> [EXTERNAL EMAIL]
>>> >>>
>>> >>> Hi Zhuzhu,
>>> >>>
>>> >>> Thanks for starting the discussion.
>>> >>>
>>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>>> regression
>>> >>> from previous versions and is currently blocking the development of
>>> >> Pravega
>>> >>> source on top of FLIP-27.
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Jiangjie (Becket) Qin
>>> >>>
>>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
>>> >>>
>>> >>>> Thank you all for the inputs!
>>> >>>>
>>> >>>> I agree with Till that we should set a soft deadline first.
>>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>>> >>>> But feel free to raise your concerns if you feel next Monday as a
>>> >>>> deadline may not work for fixes which should be a blocker for
>>> 1.11.2.
>>> >>>>
>>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>> HiveBulkWriterFactory
>>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>>> >>>> because job is not finished when dispatcher close
>>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>>> >>>> stream
>>> >>>>
>>> >>>> Thanks,
>>> >>>> Zhu
>>> >>>>
>>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>>> >>>>
>>> >>>>> I think it would be nice to include a fix for
>>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>>> >>>>>
>>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <trohrmann@apache.org
>>> >
>>> >>>> wrote:
>>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>>> >>>>>> volunteering
>>> >>>> as
>>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>>> >>>> release. I
>>> >>>>>> think we already collected quite a bit of fixes which our users
>>> >>>>>> will benefit from.
>>> >>>>>>
>>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>>> >>>>>> (maybe
>>> >>>>> until
>>> >>>>>> beginning of next week) and then starting to cut the release
>>> >>>>>> (given
>>> >>>> that
>>> >>>>> no
>>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>>> >>>>>> issues
>>> >>>>> even
>>> >>>>>> earlier which would allow us to cut the release also earlier.
>>> >>>>>>
>>> >>>>>> From my side I would like to include FLINK-18959 in the release.
>>> >>>>>> But it
>>> >>>>> is
>>> >>>>>> not a strict release blocker.
>>> >>>>>>
>>> >>>>>> Cheers,
>>> >>>>>> Till
>>> >>>>>>
>>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>>> >>>>>> <da...@alpinegizmo.com>
>>> >>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> I think it's worth considering whether we can get this bugfix
>>> >>>> included
>>> >>>>> in
>>> >>>>>>> 1.11.2:
>>> >>>>>>>
>>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>> >>>>>>>
>>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>>> >>>>>>> who
>>> >>>> has
>>> >>>>>>> been working on this.
>>> >>>>>>>
>>> >>>>>>> Regards,
>>> >>>>>>> David
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
>>> >> wrote:
>>> >>>>>>>> Hi All,
>>> >>>>>>>>
>>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>>> >>>>>>>> not
>>> >>>>> too
>>> >>>>>>> far
>>> >>>>>>>> but
>>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
>>> >>>>>>> release-1.11
>>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>>> >>>>>>>> to
>>> >>>>> create
>>> >>>>>>> the
>>> >>>>>>>> next
>>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>>> >>>>>>>>
>>> >>>>>>>> Most noticeable fixes are:
>>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>>> >>>>>>>> operations
>>> >>>> in
>>> >>>>>>>> per-job mode
>>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>>> >>>>>>>> nullability
>>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>>> >>>>>>>> thread
>>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>>> >>>>>>>> incorrectly
>>> >>>>> in
>>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>>> >>>>>>>> previous
>>> >>>>>>>> jdk8u72(-b01)
>>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>>> >>>>>>>>
>>> >>>>>>>> Furthermore, I think the following blocker issue should be
>>> >>>>>>>> merged
>>> >>>>>> before
>>> >>>>>>>> 1.11.2 release
>>> >>>>>>>>
>>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>> >>>>> HiveBulkWriterFactory
>>> >>>>>>>> I would volunteer as the release manager and kick off the
>>> >>>>>>>> release
>>> >>>>>>> process.
>>> >>>>>>>> What do you think?
>>> >>>>>>>>
>>> >>>>>>>> Please let me know if there are any concerns or any missing
>>> >>>>>>>> blocker
>>> >>>>>>> issues
>>> >>>>>>>> need to be fixed in 1.11.2.
>>> >>>>>>>>
>>> >>>>>>>> Thanks,
>>> >>>>>>>> Zhu Zhu
>>> >>>>>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>>
>>> >>>>> Konstantin Knauf
>>> >>>>>
>>> >>>>> https://twitter.com/snntrable
>>> >>>>>
>>> >>>>> https://github.com/knaufk
>>> >>>>>
>>>
>>>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
@Dawid
Thanks for the information and preparing a fix for FLINK-19133!
I have made it a blocker for 1.11.2 and will keep tracking its status.
@Till
Thanks for the updates and efforts for FLINK-18959!
Thanks,
Zhu
Till Rohrmann <tr...@apache.org> 于2020年9月4日周五 下午3:41写道：
> Fyi, FLINK-18959 has been merged into the release-1.10 branch.
>
> Cheers,
> Till
>
> On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <dw...@apache.org>
> wrote:
>
>> User has just reported another issue FLINK-19133 which I think should be
>> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
>> possible.
>>
>> On 03/09/2020 09:36, Zhu Zhu wrote:
>> > Thanks for the inputs!
>> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>> >
>> > And thanks a lot for offering help, zhijiang!
>> >
>> > Thanks,
>> > Zhu
>> >
>> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
>> >
>> >> Hi
>> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
>> fixes a
>> >> bug introduce in 1.11.0), there is a pr for it already.
>> >> Best,
>> >> Congxian
>> >>
>> >>
>> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>> >>
>> >>> Hi,
>> >>>
>> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
>> for
>> >>> Pravega connector integration, hope we can have it included in 1.11.2
>> >>> release.
>> >>>
>> >>> Best Regards,
>> >>> Brian
>> >>>
>> >>> -----Original Message-----
>> >>> From: Becket Qin <be...@gmail.com>
>> >>> Sent: Thursday, September 3, 2020 11:18
>> >>> To: dev
>> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
>> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>> >>>
>> >>>
>> >>> [EXTERNAL EMAIL]
>> >>>
>> >>> Hi Zhuzhu,
>> >>>
>> >>> Thanks for starting the discussion.
>> >>>
>> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a
>> regression
>> >>> from previous versions and is currently blocking the development of
>> >> Pravega
>> >>> source on top of FLIP-27.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Jiangjie (Becket) Qin
>> >>>
>> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
>> >>>
>> >>>> Thank you all for the inputs!
>> >>>>
>> >>>> I agree with Till that we should set a soft deadline first.
>> >>>> I'd like to propose next Monday if no new blocker issue pops up.
>> >>>> But feel free to raise your concerns if you feel next Monday as a
>> >>>> deadline may not work for fixes which should be a blocker for 1.11.2.
>> >>>>
>> >>>> Here's a summary of the wanted/blocker but still open fixes:
>> >>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> HiveBulkWriterFactory
>> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>> >>>> because job is not finished when dispatcher close
>> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
>> >>>> stream
>> >>>>
>> >>>> Thanks,
>> >>>> Zhu
>> >>>>
>> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>> >>>>
>> >>>>> I think it would be nice to include a fix for
>> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
>> >>>>>
>> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
>> >>>> wrote:
>> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>> >>>>>> volunteering
>> >>>> as
>> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>> >>>> release. I
>> >>>>>> think we already collected quite a bit of fixes which our users
>> >>>>>> will benefit from.
>> >>>>>>
>> >>>>>> For the pending fixes, I would suggest setting a soft deadline
>> >>>>>> (maybe
>> >>>>> until
>> >>>>>> beginning of next week) and then starting to cut the release
>> >>>>>> (given
>> >>>> that
>> >>>>> no
>> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
>> >>>>>> issues
>> >>>>> even
>> >>>>>> earlier which would allow us to cut the release also earlier.
>> >>>>>>
>> >>>>>> From my side I would like to include FLINK-18959 in the release.
>> >>>>>> But it
>> >>>>> is
>> >>>>>> not a strict release blocker.
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Till
>> >>>>>>
>> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>> >>>>>> <da...@alpinegizmo.com>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> I think it's worth considering whether we can get this bugfix
>> >>>> included
>> >>>>> in
>> >>>>>>> 1.11.2:
>> >>>>>>>
>> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>> >>>>>>>
>> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>> >>>>>>> who
>> >>>> has
>> >>>>>>> been working on this.
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> David
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
>> >> wrote:
>> >>>>>>>> Hi All,
>> >>>>>>>>
>> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>> >>>>>>>> not
>> >>>>> too
>> >>>>>>> far
>> >>>>>>>> but
>> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
>> >>>>>>> release-1.11
>> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>> >>>>>>>> to
>> >>>>> create
>> >>>>>>> the
>> >>>>>>>> next
>> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>> >>>>>>>>
>> >>>>>>>> Most noticeable fixes are:
>> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>> >>>>>>>> operations
>> >>>> in
>> >>>>>>>> per-job mode
>> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>> >>>>>>>> nullability
>> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>> >>>>>>>> thread
>> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>> >>>>>>>> incorrectly
>> >>>>> in
>> >>>>>>>> JobManagerFlinkMemoryUtils.java
>> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
>> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>> >>>>>>>> previous
>> >>>>>>>> jdk8u72(-b01)
>> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>> >>>>>>>>
>> >>>>>>>> Furthermore, I think the following blocker issue should be
>> >>>>>>>> merged
>> >>>>>> before
>> >>>>>>>> 1.11.2 release
>> >>>>>>>>
>> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>> >>>>> HiveBulkWriterFactory
>> >>>>>>>> I would volunteer as the release manager and kick off the
>> >>>>>>>> release
>> >>>>>>> process.
>> >>>>>>>> What do you think?
>> >>>>>>>>
>> >>>>>>>> Please let me know if there are any concerns or any missing
>> >>>>>>>> blocker
>> >>>>>>> issues
>> >>>>>>>> need to be fixed in 1.11.2.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> Zhu Zhu
>> >>>>>>>>
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> Konstantin Knauf
>> >>>>>
>> >>>>> https://twitter.com/snntrable
>> >>>>>
>> >>>>> https://github.com/knaufk
>> >>>>>
>>
>>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Till Rohrmann <tr...@apache.org>.
Fyi, FLINK-18959 has been merged into the release-1.10 branch.
Cheers,
Till
On Thu, Sep 3, 2020 at 2:38 PM Dawid Wysakowicz <dw...@apache.org>
wrote:
> User has just reported another issue FLINK-19133 which I think should be
> a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
> possible.
>
> On 03/09/2020 09:36, Zhu Zhu wrote:
> > Thanks for the inputs!
> > I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
> >
> > And thanks a lot for offering help, zhijiang!
> >
> > Thanks,
> > Zhu
> >
> > Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
> >
> >> Hi
> >> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this
> fixes a
> >> bug introduce in 1.11.0), there is a pr for it already.
> >> Best,
> >> Congxian
> >>
> >>
> >> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
> >>
> >>> Hi,
> >>>
> >>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker
> for
> >>> Pravega connector integration, hope we can have it included in 1.11.2
> >>> release.
> >>>
> >>> Best Regards,
> >>> Brian
> >>>
> >>> -----Original Message-----
> >>> From: Becket Qin <be...@gmail.com>
> >>> Sent: Thursday, September 3, 2020 11:18
> >>> To: dev
> >>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
> >>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >>>
> >>>
> >>> [EXTERNAL EMAIL]
> >>>
> >>> Hi Zhuzhu,
> >>>
> >>> Thanks for starting the discussion.
> >>>
> >>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> >>> from previous versions and is currently blocking the development of
> >> Pravega
> >>> source on top of FLIP-27.
> >>>
> >>> Thanks,
> >>>
> >>> Jiangjie (Becket) Qin
> >>>
> >>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
> >>>
> >>>> Thank you all for the inputs!
> >>>>
> >>>> I agree with Till that we should set a soft deadline first.
> >>>> I'd like to propose next Monday if no new blocker issue pops up.
> >>>> But feel free to raise your concerns if you feel next Monday as a
> >>>> deadline may not work for fixes which should be a blocker for 1.11.2.
> >>>>
> >>>> Here's a summary of the wanted/blocker but still open fixes:
> >>>> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> >>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> >>>> because job is not finished when dispatcher close
> >>>> - FLINK-18934 Idle stream does not advance watermark in connected
> >>>> stream
> >>>>
> >>>> Thanks,
> >>>> Zhu
> >>>>
> >>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> >>>>
> >>>>> I think it would be nice to include a fix for
> >>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> >>>>> affects a highly requested feature of Flink 1.11 quite severely.
> >>>>>
> >>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
> >>>> wrote:
> >>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
> >>>>>> volunteering
> >>>> as
> >>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
> >>>> release. I
> >>>>>> think we already collected quite a bit of fixes which our users
> >>>>>> will benefit from.
> >>>>>>
> >>>>>> For the pending fixes, I would suggest setting a soft deadline
> >>>>>> (maybe
> >>>>> until
> >>>>>> beginning of next week) and then starting to cut the release
> >>>>>> (given
> >>>> that
> >>>>> no
> >>>>>> other blocker issues pop up). Maybe we are able to resolve the
> >>>>>> issues
> >>>>> even
> >>>>>> earlier which would allow us to cut the release also earlier.
> >>>>>>
> >>>>>> From my side I would like to include FLINK-18959 in the release.
> >>>>>> But it
> >>>>> is
> >>>>>> not a strict release blocker.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Till
> >>>>>>
> >>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> >>>>>> <da...@alpinegizmo.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I think it's worth considering whether we can get this bugfix
> >>>> included
> >>>>> in
> >>>>>>> 1.11.2:
> >>>>>>>
> >>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
> >>>>>>>
> >>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
> >>>>>>> who
> >>>> has
> >>>>>>> been working on this.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> David
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
> >> wrote:
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
> >>>>>>>> not
> >>>>> too
> >>>>>>> far
> >>>>>>>> but
> >>>>>>>> we already have more than 80 resolved improvements/bugs in the
> >>>>>>> release-1.11
> >>>>>>>> branch. Some of them are quite critical. Therefore, I propose
> >>>>>>>> to
> >>>>> create
> >>>>>>> the
> >>>>>>>> next
> >>>>>>>> bugfix release 1.11.2 for Flink 1.11.
> >>>>>>>>
> >>>>>>>> Most noticeable fixes are:
> >>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> >>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
> >>>>>>>> operations
> >>>> in
> >>>>>>>> per-job mode
> >>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> >>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
> >>>>>>>> nullability
> >>>>>>>> - FLINK-18646 Managed memory released check can block RPC
> >>>>>>>> thread
> >>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> >>>>>>>> incorrectly
> >>>>> in
> >>>>>>>> JobManagerFlinkMemoryUtils.java
> >>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> >>>>>>>> - FLINK-18595 Deadlock during job shutdown
> >>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
> >>>>>>>> previous
> >>>>>>>> jdk8u72(-b01)
> >>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
> >>>>>>>>
> >>>>>>>> Furthermore, I think the following blocker issue should be
> >>>>>>>> merged
> >>>>>> before
> >>>>>>>> 1.11.2 release
> >>>>>>>>
> >>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
> >>>>> HiveBulkWriterFactory
> >>>>>>>> I would volunteer as the release manager and kick off the
> >>>>>>>> release
> >>>>>>> process.
> >>>>>>>> What do you think?
> >>>>>>>>
> >>>>>>>> Please let me know if there are any concerns or any missing
> >>>>>>>> blocker
> >>>>>>> issues
> >>>>>>>> need to be fixed in 1.11.2.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Zhu Zhu
> >>>>>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Konstantin Knauf
> >>>>>
> >>>>> https://twitter.com/snntrable
> >>>>>
> >>>>> https://github.com/knaufk
> >>>>>
>
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Dawid Wysakowicz <dw...@apache.org>.
User has just reported another issue FLINK-19133 which I think should be
a blocker for the 1.11.2 release. I'll try to prepare a fix as soon as
possible.
On 03/09/2020 09:36, Zhu Zhu wrote:
> Thanks for the inputs!
> I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
>
> And thanks a lot for offering help, zhijiang!
>
> Thanks,
> Zhu
>
> Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
>
>> Hi
>> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
>> bug introduce in 1.11.0), there is a pr for it already.
>> Best,
>> Congxian
>>
>>
>> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>>
>>> Hi,
>>>
>>> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
>>> Pravega connector integration, hope we can have it included in 1.11.2
>>> release.
>>>
>>> Best Regards,
>>> Brian
>>>
>>> -----Original Message-----
>>> From: Becket Qin <be...@gmail.com>
>>> Sent: Thursday, September 3, 2020 11:18
>>> To: dev
>>> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
>>> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>>>
>>>
>>> [EXTERNAL EMAIL]
>>>
>>> Hi Zhuzhu,
>>>
>>> Thanks for starting the discussion.
>>>
>>> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
>>> from previous versions and is currently blocking the development of
>> Pravega
>>> source on top of FLIP-27.
>>>
>>> Thanks,
>>>
>>> Jiangjie (Becket) Qin
>>>
>>> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
>>>
>>>> Thank you all for the inputs!
>>>>
>>>> I agree with Till that we should set a soft deadline first.
>>>> I'd like to propose next Monday if no new blocker issue pops up.
>>>> But feel free to raise your concerns if you feel next Monday as a
>>>> deadline may not work for fixes which should be a blocker for 1.11.2.
>>>>
>>>> Here's a summary of the wanted/blocker but still open fixes:
>>>> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>>> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
>>>> because job is not finished when dispatcher close
>>>> - FLINK-18934 Idle stream does not advance watermark in connected
>>>> stream
>>>>
>>>> Thanks,
>>>> Zhu
>>>>
>>>> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>>>>
>>>>> I think it would be nice to include a fix for
>>>>> https://issues.apache.org/jira/browse/FLINK-18934, too, as it
>>>>> affects a highly requested feature of Flink 1.11 quite severely.
>>>>>
>>>>> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
>>>> wrote:
>>>>>> Thanks a lot for starting this discussion Zhu Zhu and for
>>>>>> volunteering
>>>> as
>>>>>> the release manager. Big +1 for creating the next 1.11 bug fix
>>>> release. I
>>>>>> think we already collected quite a bit of fixes which our users
>>>>>> will benefit from.
>>>>>>
>>>>>> For the pending fixes, I would suggest setting a soft deadline
>>>>>> (maybe
>>>>> until
>>>>>> beginning of next week) and then starting to cut the release
>>>>>> (given
>>>> that
>>>>> no
>>>>>> other blocker issues pop up). Maybe we are able to resolve the
>>>>>> issues
>>>>> even
>>>>>> earlier which would allow us to cut the release also earlier.
>>>>>>
>>>>>> From my side I would like to include FLINK-18959 in the release.
>>>>>> But it
>>>>> is
>>>>>> not a strict release blocker.
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Wed, Sep 2, 2020 at 2:40 PM David Anderson
>>>>>> <da...@alpinegizmo.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I think it's worth considering whether we can get this bugfix
>>>> included
>>>>> in
>>>>>>> 1.11.2:
>>>>>>>
>>>>>>> - FLINK-19109 Split Reader eats chained periodic watermarks
>>>>>>>
>>>>>>> There is a PR, but it's still a work in progress. Cc'ing Roman,
>>>>>>> who
>>>> has
>>>>>>> been working on this.
>>>>>>>
>>>>>>> Regards,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
>> wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> It has been about 1 month since we released Flink 1.11.1. It's
>>>>>>>> not
>>>>> too
>>>>>>> far
>>>>>>>> but
>>>>>>>> we already have more than 80 resolved improvements/bugs in the
>>>>>>> release-1.11
>>>>>>>> branch. Some of them are quite critical. Therefore, I propose
>>>>>>>> to
>>>>> create
>>>>>>> the
>>>>>>>> next
>>>>>>>> bugfix release 1.11.2 for Flink 1.11.
>>>>>>>>
>>>>>>>> Most noticeable fixes are:
>>>>>>>> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
>>>>>>>> - FLINK-18902 Cannot serve results of asynchronous REST
>>>>>>>> operations
>>>> in
>>>>>>>> per-job mode
>>>>>>>> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
>>>>>>>> - FLINK-18608 CustomizedConvertRule#convertCast drops
>>>>>>>> nullability
>>>>>>>> - FLINK-18646 Managed memory released check can block RPC
>>>>>>>> thread
>>>>>>>> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
>>>>>>>> incorrectly
>>>>> in
>>>>>>>> JobManagerFlinkMemoryUtils.java
>>>>>>>> - FLINK-18663 RestServerEndpoint may prevent server shutdown
>>>>>>>> - FLINK-18595 Deadlock during job shutdown
>>>>>>>> - FLINK-18581 Cannot find GC cleaner with java version
>>>>>>>> previous
>>>>>>>> jdk8u72(-b01)
>>>>>>>> - FLINK-17075 Add task status reconciliation between TM and JM
>>>>>>>>
>>>>>>>> Furthermore, I think the following blocker issue should be
>>>>>>>> merged
>>>>>> before
>>>>>>>> 1.11.2 release
>>>>>>>>
>>>>>>>> - FLINK-19121 Avoid accessing HDFS frequently in
>>>>> HiveBulkWriterFactory
>>>>>>>> I would volunteer as the release manager and kick off the
>>>>>>>> release
>>>>>>> process.
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Please let me know if there are any concerns or any missing
>>>>>>>> blocker
>>>>>>> issues
>>>>>>>> need to be fixed in 1.11.2.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Zhu Zhu
>>>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Konstantin Knauf
>>>>>
>>>>> https://twitter.com/snntrable
>>>>>
>>>>> https://github.com/knaufk
>>>>>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
Thanks for the inputs!
I have made FLINK-14942 and FLINK-18641 blockers for 1.11.2.
And thanks a lot for offering help, zhijiang!
Thanks,
Zhu
Congxian Qiu <qc...@gmail.com> 于2020年9月3日周四 下午3:18写道：
> Hi
> I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
> bug introduce in 1.11.0), there is a pr for it already.
> Best,
> Congxian
>
>
> Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
>
> > Hi,
> >
> > Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
> > Pravega connector integration, hope we can have it included in 1.11.2
> > release.
> >
> > Best Regards,
> > Brian
> >
> > -----Original Message-----
> > From: Becket Qin <be...@gmail.com>
> > Sent: Thursday, September 3, 2020 11:18
> > To: dev
> > Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
> > Subject: Re: [DISCUSS] Releasing Flink 1.11.2
> >
> >
> > [EXTERNAL EMAIL]
> >
> > Hi Zhuzhu,
> >
> > Thanks for starting the discussion.
> >
> > I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> > from previous versions and is currently blocking the development of
> Pravega
> > source on top of FLIP-27.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
> >
> > > Thank you all for the inputs!
> > >
> > > I agree with Till that we should set a soft deadline first.
> > > I'd like to propose next Monday if no new blocker issue pops up.
> > > But feel free to raise your concerns if you feel next Monday as a
> > > deadline may not work for fixes which should be a blocker for 1.11.2.
> > >
> > > Here's a summary of the wanted/blocker but still open fixes:
> > > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> > > because job is not finished when dispatcher close
> > > - FLINK-18934 Idle stream does not advance watermark in connected
> > > stream
> > >
> > > Thanks,
> > > Zhu
> > >
> > > Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> > >
> > > > I think it would be nice to include a fix for
> > > > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > > > affects a highly requested feature of Flink 1.11 quite severely.
> > > >
> > > > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
> > > wrote:
> > > >
> > > > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > > > volunteering
> > > as
> > > > > the release manager. Big +1 for creating the next 1.11 bug fix
> > > release. I
> > > > > think we already collected quite a bit of fixes which our users
> > > > > will benefit from.
> > > > >
> > > > > For the pending fixes, I would suggest setting a soft deadline
> > > > > (maybe
> > > > until
> > > > > beginning of next week) and then starting to cut the release
> > > > > (given
> > > that
> > > > no
> > > > > other blocker issues pop up). Maybe we are able to resolve the
> > > > > issues
> > > > even
> > > > > earlier which would allow us to cut the release also earlier.
> > > > >
> > > > > From my side I would like to include FLINK-18959 in the release.
> > > > > But it
> > > > is
> > > > > not a strict release blocker.
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > > > <da...@alpinegizmo.com>
> > > > > wrote:
> > > > >
> > > > > > I think it's worth considering whether we can get this bugfix
> > > included
> > > > in
> > > > > > 1.11.2:
> > > > > >
> > > > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > > > >
> > > > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > > > who
> > > has
> > > > > > been working on this.
> > > > > >
> > > > > > Regards,
> > > > > > David
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com>
> wrote:
> > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > > > not
> > > > too
> > > > > > far
> > > > > > > but
> > > > > > > we already have more than 80 resolved improvements/bugs in the
> > > > > > release-1.11
> > > > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > > > to
> > > > create
> > > > > > the
> > > > > > > next
> > > > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > > > >
> > > > > > > Most noticeable fixes are:
> > > > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > > > operations
> > > in
> > > > > > > per-job mode
> > > > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > > > nullability
> > > > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > > > thread
> > > > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > > > incorrectly
> > > > in
> > > > > > > JobManagerFlinkMemoryUtils.java
> > > > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > > > previous
> > > > > > > jdk8u72(-b01)
> > > > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > > > >
> > > > > > > Furthermore, I think the following blocker issue should be
> > > > > > > merged
> > > > > before
> > > > > > > 1.11.2 release
> > > > > > >
> > > > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > > > HiveBulkWriterFactory
> > > > > > >
> > > > > > > I would volunteer as the release manager and kick off the
> > > > > > > release
> > > > > > process.
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Please let me know if there are any concerns or any missing
> > > > > > > blocker
> > > > > > issues
> > > > > > > need to be fixed in 1.11.2.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Zhu Zhu
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Konstantin Knauf
> > > >
> > > > https://twitter.com/snntrable
> > > >
> > > > https://github.com/knaufk
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Congxian Qiu <qc...@gmail.com>.
Hi
I'd like to include FLINK-14942 into 1.11.2. FLINK-14942 (this fixes a
bug introduce in 1.11.0), there is a pr for it already.
Best,
Congxian
Zhou, Brian <B....@dell.com> 于2020年9月3日周四 上午11:21写道：
> Hi,
>
> Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for
> Pravega connector integration, hope we can have it included in 1.11.2
> release.
>
> Best Regards,
> Brian
>
> -----Original Message-----
> From: Becket Qin <be...@gmail.com>
> Sent: Thursday, September 3, 2020 11:18
> To: dev
> Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
> Subject: Re: [DISCUSS] Releasing Flink 1.11.2
>
>
> [EXTERNAL EMAIL]
>
> Hi Zhuzhu,
>
> Thanks for starting the discussion.
>
> I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
> from previous versions and is currently blocking the development of Pravega
> source on top of FLIP-27.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
>
> > Thank you all for the inputs!
> >
> > I agree with Till that we should set a soft deadline first.
> > I'd like to propose next Monday if no new blocker issue pops up.
> > But feel free to raise your concerns if you feel next Monday as a
> > deadline may not work for fixes which should be a blocker for 1.11.2.
> >
> > Here's a summary of the wanted/blocker but still open fixes:
> > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > - FLINK-19109 Split Reader eats chained periodic watermarks
> > - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> > because job is not finished when dispatcher close
> > - FLINK-18934 Idle stream does not advance watermark in connected
> > stream
> >
> > Thanks,
> > Zhu
> >
> > Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> >
> > > I think it would be nice to include a fix for
> > > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > > affects a highly requested feature of Flink 1.11 quite severely.
> > >
> > > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
> > wrote:
> > >
> > > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > > volunteering
> > as
> > > > the release manager. Big +1 for creating the next 1.11 bug fix
> > release. I
> > > > think we already collected quite a bit of fixes which our users
> > > > will benefit from.
> > > >
> > > > For the pending fixes, I would suggest setting a soft deadline
> > > > (maybe
> > > until
> > > > beginning of next week) and then starting to cut the release
> > > > (given
> > that
> > > no
> > > > other blocker issues pop up). Maybe we are able to resolve the
> > > > issues
> > > even
> > > > earlier which would allow us to cut the release also earlier.
> > > >
> > > > From my side I would like to include FLINK-18959 in the release.
> > > > But it
> > > is
> > > > not a strict release blocker.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > > <da...@alpinegizmo.com>
> > > > wrote:
> > > >
> > > > > I think it's worth considering whether we can get this bugfix
> > included
> > > in
> > > > > 1.11.2:
> > > > >
> > > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > > >
> > > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > > who
> > has
> > > > > been working on this.
> > > > >
> > > > > Regards,
> > > > > David
> > > > >
> > > > >
> > > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > > not
> > > too
> > > > > far
> > > > > > but
> > > > > > we already have more than 80 resolved improvements/bugs in the
> > > > > release-1.11
> > > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > > to
> > > create
> > > > > the
> > > > > > next
> > > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > > >
> > > > > > Most noticeable fixes are:
> > > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > > operations
> > in
> > > > > > per-job mode
> > > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > > nullability
> > > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > > thread
> > > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > > incorrectly
> > > in
> > > > > > JobManagerFlinkMemoryUtils.java
> > > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > > previous
> > > > > > jdk8u72(-b01)
> > > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > > >
> > > > > > Furthermore, I think the following blocker issue should be
> > > > > > merged
> > > > before
> > > > > > 1.11.2 release
> > > > > >
> > > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > > HiveBulkWriterFactory
> > > > > >
> > > > > > I would volunteer as the release manager and kick off the
> > > > > > release
> > > > > process.
> > > > > > What do you think?
> > > > > >
> > > > > > Please let me know if there are any concerns or any missing
> > > > > > blocker
> > > > > issues
> > > > > > need to be fixed in 1.11.2.
> > > > > >
> > > > > > Thanks,
> > > > > > Zhu Zhu
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>
RE: [DISCUSS] Releasing Flink 1.11.2
Posted by "Zhou, Brian" <B....@dell.com>.
Hi,
Thanks Becket for addressing the issue. FLINK-18641 is now a blocker for Pravega connector integration, hope we can have it included in 1.11.2 release.
Best Regards,
Brian
-----Original Message-----
From: Becket Qin <be...@gmail.com>
Sent: Thursday, September 3, 2020 11:18
To: dev
Cc: khachatryan.roman@gmail.com; Till Rohrmann; david; Jingsong Li
Subject: Re: [DISCUSS] Releasing Flink 1.11.2
[EXTERNAL EMAIL]
Hi Zhuzhu,
Thanks for starting the discussion.
I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression from previous versions and is currently blocking the development of Pravega source on top of FLIP-27.
Thanks,
Jiangjie (Becket) Qin
On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
> Thank you all for the inputs!
>
> I agree with Till that we should set a soft deadline first.
> I'd like to propose next Monday if no new blocker issue pops up.
> But feel free to raise your concerns if you feel next Monday as a
> deadline may not work for fixes which should be a blocker for 1.11.2.
>
> Here's a summary of the wanted/blocker but still open fixes:
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> - FLINK-19109 Split Reader eats chained periodic watermarks
> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph
> because job is not finished when dispatcher close
> - FLINK-18934 Idle stream does not advance watermark in connected
> stream
>
> Thanks,
> Zhu
>
> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>
> > I think it would be nice to include a fix for
> > https://issues.apache.org/jira/browse/FLINK-18934, too, as it
> > affects a highly requested feature of Flink 1.11 quite severely.
> >
> > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> > > Thanks a lot for starting this discussion Zhu Zhu and for
> > > volunteering
> as
> > > the release manager. Big +1 for creating the next 1.11 bug fix
> release. I
> > > think we already collected quite a bit of fixes which our users
> > > will benefit from.
> > >
> > > For the pending fixes, I would suggest setting a soft deadline
> > > (maybe
> > until
> > > beginning of next week) and then starting to cut the release
> > > (given
> that
> > no
> > > other blocker issues pop up). Maybe we are able to resolve the
> > > issues
> > even
> > > earlier which would allow us to cut the release also earlier.
> > >
> > > From my side I would like to include FLINK-18959 in the release.
> > > But it
> > is
> > > not a strict release blocker.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson
> > > <da...@alpinegizmo.com>
> > > wrote:
> > >
> > > > I think it's worth considering whether we can get this bugfix
> included
> > in
> > > > 1.11.2:
> > > >
> > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > >
> > > > There is a PR, but it's still a work in progress. Cc'ing Roman,
> > > > who
> has
> > > > been working on this.
> > > >
> > > > Regards,
> > > > David
> > > >
> > > >
> > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > It has been about 1 month since we released Flink 1.11.1. It's
> > > > > not
> > too
> > > > far
> > > > > but
> > > > > we already have more than 80 resolved improvements/bugs in the
> > > > release-1.11
> > > > > branch. Some of them are quite critical. Therefore, I propose
> > > > > to
> > create
> > > > the
> > > > > next
> > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > >
> > > > > Most noticeable fixes are:
> > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > - FLINK-18902 Cannot serve results of asynchronous REST
> > > > > operations
> in
> > > > > per-job mode
> > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops
> > > > > nullability
> > > > > - FLINK-18646 Managed memory released check can block RPC
> > > > > thread
> > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method
> > > > > incorrectly
> > in
> > > > > JobManagerFlinkMemoryUtils.java
> > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > - FLINK-18581 Cannot find GC cleaner with java version
> > > > > previous
> > > > > jdk8u72(-b01)
> > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > >
> > > > > Furthermore, I think the following blocker issue should be
> > > > > merged
> > > before
> > > > > 1.11.2 release
> > > > >
> > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > HiveBulkWriterFactory
> > > > >
> > > > > I would volunteer as the release manager and kick off the
> > > > > release
> > > > process.
> > > > > What do you think?
> > > > >
> > > > > Please let me know if there are any concerns or any missing
> > > > > blocker
> > > > issues
> > > > > need to be fixed in 1.11.2.
> > > > >
> > > > > Thanks,
> > > > > Zhu Zhu
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Becket Qin <be...@gmail.com>.
Hi Zhuzhu,
Thanks for starting the discussion.
I'd like to include FLINK-18641 into 1.11.2 as well. It is a regression
from previous versions and is currently blocking the development of Pravega
source on top of FLIP-27.
Thanks,
Jiangjie (Becket) Qin
On Wed, Sep 2, 2020 at 11:13 PM Zhu Zhu <re...@gmail.com> wrote:
> Thank you all for the inputs!
>
> I agree with Till that we should set a soft deadline first.
> I'd like to propose next Monday if no new blocker issue pops up.
> But feel free to raise your concerns if you feel next Monday as a deadline
> may not work
> for fixes which should be a blocker for 1.11.2.
>
> Here's a summary of the wanted/blocker but still open fixes:
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> - FLINK-19109 Split Reader eats chained periodic watermarks
> - (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph because
> job is not finished when dispatcher close
> - FLINK-18934 Idle stream does not advance watermark in connected stream
>
> Thanks,
> Zhu
>
> Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
>
> > I think it would be nice to include a fix for
> > https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
> > highly requested feature of Flink 1.11 quite severely.
> >
> > On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org>
> wrote:
> >
> > > Thanks a lot for starting this discussion Zhu Zhu and for volunteering
> as
> > > the release manager. Big +1 for creating the next 1.11 bug fix
> release. I
> > > think we already collected quite a bit of fixes which our users will
> > > benefit from.
> > >
> > > For the pending fixes, I would suggest setting a soft deadline (maybe
> > until
> > > beginning of next week) and then starting to cut the release (given
> that
> > no
> > > other blocker issues pop up). Maybe we are able to resolve the issues
> > even
> > > earlier which would allow us to cut the release also earlier.
> > >
> > > From my side I would like to include FLINK-18959 in the release. But it
> > is
> > > not a strict release blocker.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Sep 2, 2020 at 2:40 PM David Anderson <da...@alpinegizmo.com>
> > > wrote:
> > >
> > > > I think it's worth considering whether we can get this bugfix
> included
> > in
> > > > 1.11.2:
> > > >
> > > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > > >
> > > > There is a PR, but it's still a work in progress. Cc'ing Roman, who
> has
> > > > been working on this.
> > > >
> > > > Regards,
> > > > David
> > > >
> > > >
> > > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > It has been about 1 month since we released Flink 1.11.1. It's not
> > too
> > > > far
> > > > > but
> > > > > we already have more than 80 resolved improvements/bugs in the
> > > > release-1.11
> > > > > branch. Some of them are quite critical. Therefore, I propose to
> > create
> > > > the
> > > > > next
> > > > > bugfix release 1.11.2 for Flink 1.11.
> > > > >
> > > > > Most noticeable fixes are:
> > > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > > - FLINK-18902 Cannot serve results of asynchronous REST operations
> in
> > > > > per-job mode
> > > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > > > - FLINK-18646 Managed memory released check can block RPC thread
> > > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly
> > in
> > > > > JobManagerFlinkMemoryUtils.java
> > > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > > - FLINK-18595 Deadlock during job shutdown
> > > > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > > > jdk8u72(-b01)
> > > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > > >
> > > > > Furthermore, I think the following blocker issue should be merged
> > > before
> > > > > 1.11.2 release
> > > > >
> > > > > - FLINK-19121 Avoid accessing HDFS frequently in
> > HiveBulkWriterFactory
> > > > >
> > > > > I would volunteer as the release manager and kick off the release
> > > > process.
> > > > > What do you think?
> > > > >
> > > > > Please let me know if there are any concerns or any missing blocker
> > > > issues
> > > > > need to be fixed in 1.11.2.
> > > > >
> > > > > Thanks,
> > > > > Zhu Zhu
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhu Zhu <re...@gmail.com>.
Thank you all for the inputs!
I agree with Till that we should set a soft deadline first.
I'd like to propose next Monday if no new blocker issue pops up.
But feel free to raise your concerns if you feel next Monday as a deadline
may not work
for fixes which should be a blocker for 1.11.2.
Here's a summary of the wanted/blocker but still open fixes:
- FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
- FLINK-19109 Split Reader eats chained periodic watermarks
- (not a strict blocker) FLINK-18959 Fail to archiveExecutionGraph because
job is not finished when dispatcher close
- FLINK-18934 Idle stream does not advance watermark in connected stream
Thanks,
Zhu
Konstantin Knauf <kn...@apache.org> 于2020年9月2日周三 下午9:00写道：
> I think it would be nice to include a fix for
> https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
> highly requested feature of Flink 1.11 quite severely.
>
> On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org> wrote:
>
> > Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
> > the release manager. Big +1 for creating the next 1.11 bug fix release. I
> > think we already collected quite a bit of fixes which our users will
> > benefit from.
> >
> > For the pending fixes, I would suggest setting a soft deadline (maybe
> until
> > beginning of next week) and then starting to cut the release (given that
> no
> > other blocker issues pop up). Maybe we are able to resolve the issues
> even
> > earlier which would allow us to cut the release also earlier.
> >
> > From my side I would like to include FLINK-18959 in the release. But it
> is
> > not a strict release blocker.
> >
> > Cheers,
> > Till
> >
> > On Wed, Sep 2, 2020 at 2:40 PM David Anderson <da...@alpinegizmo.com>
> > wrote:
> >
> > > I think it's worth considering whether we can get this bugfix included
> in
> > > 1.11.2:
> > >
> > > - FLINK-19109 Split Reader eats chained periodic watermarks
> > >
> > > There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> > > been working on this.
> > >
> > > Regards,
> > > David
> > >
> > >
> > > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> > >
> > > > Hi All,
> > > >
> > > > It has been about 1 month since we released Flink 1.11.1. It's not
> too
> > > far
> > > > but
> > > > we already have more than 80 resolved improvements/bugs in the
> > > release-1.11
> > > > branch. Some of them are quite critical. Therefore, I propose to
> create
> > > the
> > > > next
> > > > bugfix release 1.11.2 for Flink 1.11.
> > > >
> > > > Most noticeable fixes are:
> > > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > > > per-job mode
> > > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > > - FLINK-18646 Managed memory released check can block RPC thread
> > > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly
> in
> > > > JobManagerFlinkMemoryUtils.java
> > > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > > - FLINK-18595 Deadlock during job shutdown
> > > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > > jdk8u72(-b01)
> > > > - FLINK-17075 Add task status reconciliation between TM and JM
> > > >
> > > > Furthermore, I think the following blocker issue should be merged
> > before
> > > > 1.11.2 release
> > > >
> > > > - FLINK-19121 Avoid accessing HDFS frequently in
> HiveBulkWriterFactory
> > > >
> > > > I would volunteer as the release manager and kick off the release
> > > process.
> > > > What do you think?
> > > >
> > > > Please let me know if there are any concerns or any missing blocker
> > > issues
> > > > need to be fixed in 1.11.2.
> > > >
> > > > Thanks,
> > > > Zhu Zhu
> > > >
> > >
> >
>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Zhijiang <wa...@aliyun.com.INVALID>.
Thanks for launching this discussion and volunteering as the release manager.
+1 on my side and I am willing to provide any help during the release procedure, :)
Best,
Zhijiang
------------------------------------------------------------------
From:Konstantin Knauf <kn...@apache.org>
Send Time:2020年9月2日(星期三) 23:44
To:dev <de...@flink.apache.org>
Cc:khachatryan.roman <kh...@gmail.com>
Subject:Re: [DISCUSS] Releasing Flink 1.11.2
I think it would be nice to include a fix for
https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
highly requested feature of Flink 1.11 quite severely.
On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org> wrote:
> Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
> the release manager. Big +1 for creating the next 1.11 bug fix release. I
> think we already collected quite a bit of fixes which our users will
> benefit from.
>
> For the pending fixes, I would suggest setting a soft deadline (maybe until
> beginning of next week) and then starting to cut the release (given that no
> other blocker issues pop up). Maybe we are able to resolve the issues even
> earlier which would allow us to cut the release also earlier.
>
> From my side I would like to include FLINK-18959 in the release. But it is
> not a strict release blocker.
>
> Cheers,
> Till
>
> On Wed, Sep 2, 2020 at 2:40 PM David Anderson <da...@alpinegizmo.com>
> wrote:
>
> > I think it's worth considering whether we can get this bugfix included in
> > 1.11.2:
> >
> > - FLINK-19109 Split Reader eats chained periodic watermarks
> >
> > There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> > been working on this.
> >
> > Regards,
> > David
> >
> >
> > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > It has been about 1 month since we released Flink 1.11.1. It's not too
> > far
> > > but
> > > we already have more than 80 resolved improvements/bugs in the
> > release-1.11
> > > branch. Some of them are quite critical. Therefore, I propose to create
> > the
> > > next
> > > bugfix release 1.11.2 for Flink 1.11.
> > >
> > > Most noticeable fixes are:
> > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > > per-job mode
> > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > - FLINK-18646 Managed memory released check can block RPC thread
> > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> > > JobManagerFlinkMemoryUtils.java
> > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > - FLINK-18595 Deadlock during job shutdown
> > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > jdk8u72(-b01)
> > > - FLINK-17075 Add task status reconciliation between TM and JM
> > >
> > > Furthermore, I think the following blocker issue should be merged
> before
> > > 1.11.2 release
> > >
> > > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > >
> > > I would volunteer as the release manager and kick off the release
> > process.
> > > What do you think?
> > >
> > > Please let me know if there are any concerns or any missing blocker
> > issues
> > > need to be fixed in 1.11.2.
> > >
> > > Thanks,
> > > Zhu Zhu
> > >
> >
>
--
Konstantin Knauf
https://twitter.com/snntrable
https://github.com/knaufk
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Konstantin Knauf <kn...@apache.org>.
I think it would be nice to include a fix for
https://issues.apache.org/jira/browse/FLINK-18934, too, as it affects a
highly requested feature of Flink 1.11 quite severely.
On Wed, Sep 2, 2020 at 2:51 PM Till Rohrmann <tr...@apache.org> wrote:
> Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
> the release manager. Big +1 for creating the next 1.11 bug fix release. I
> think we already collected quite a bit of fixes which our users will
> benefit from.
>
> For the pending fixes, I would suggest setting a soft deadline (maybe until
> beginning of next week) and then starting to cut the release (given that no
> other blocker issues pop up). Maybe we are able to resolve the issues even
> earlier which would allow us to cut the release also earlier.
>
> From my side I would like to include FLINK-18959 in the release. But it is
> not a strict release blocker.
>
> Cheers,
> Till
>
> On Wed, Sep 2, 2020 at 2:40 PM David Anderson <da...@alpinegizmo.com>
> wrote:
>
> > I think it's worth considering whether we can get this bugfix included in
> > 1.11.2:
> >
> > - FLINK-19109 Split Reader eats chained periodic watermarks
> >
> > There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> > been working on this.
> >
> > Regards,
> > David
> >
> >
> > On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > It has been about 1 month since we released Flink 1.11.1. It's not too
> > far
> > > but
> > > we already have more than 80 resolved improvements/bugs in the
> > release-1.11
> > > branch. Some of them are quite critical. Therefore, I propose to create
> > the
> > > next
> > > bugfix release 1.11.2 for Flink 1.11.
> > >
> > > Most noticeable fixes are:
> > > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > > per-job mode
> > > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > > - FLINK-18646 Managed memory released check can block RPC thread
> > > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> > > JobManagerFlinkMemoryUtils.java
> > > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > > - FLINK-18595 Deadlock during job shutdown
> > > - FLINK-18581 Cannot find GC cleaner with java version previous
> > > jdk8u72(-b01)
> > > - FLINK-17075 Add task status reconciliation between TM and JM
> > >
> > > Furthermore, I think the following blocker issue should be merged
> before
> > > 1.11.2 release
> > >
> > > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> > >
> > > I would volunteer as the release manager and kick off the release
> > process.
> > > What do you think?
> > >
> > > Please let me know if there are any concerns or any missing blocker
> > issues
> > > need to be fixed in 1.11.2.
> > >
> > > Thanks,
> > > Zhu Zhu
> > >
> >
>
--
Konstantin Knauf
https://twitter.com/snntrable
https://github.com/knaufk
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by Till Rohrmann <tr...@apache.org>.
Thanks a lot for starting this discussion Zhu Zhu and for volunteering as
the release manager. Big +1 for creating the next 1.11 bug fix release. I
think we already collected quite a bit of fixes which our users will
benefit from.
For the pending fixes, I would suggest setting a soft deadline (maybe until
beginning of next week) and then starting to cut the release (given that no
other blocker issues pop up). Maybe we are able to resolve the issues even
earlier which would allow us to cut the release also earlier.
From my side I would like to include FLINK-18959 in the release. But it is
not a strict release blocker.
Cheers,
Till
On Wed, Sep 2, 2020 at 2:40 PM David Anderson <da...@alpinegizmo.com> wrote:
> I think it's worth considering whether we can get this bugfix included in
> 1.11.2:
>
> - FLINK-19109 Split Reader eats chained periodic watermarks
>
> There is a PR, but it's still a work in progress. Cc'ing Roman, who has
> been working on this.
>
> Regards,
> David
>
>
> On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
>
> > Hi All,
> >
> > It has been about 1 month since we released Flink 1.11.1. It's not too
> far
> > but
> > we already have more than 80 resolved improvements/bugs in the
> release-1.11
> > branch. Some of them are quite critical. Therefore, I propose to create
> the
> > next
> > bugfix release 1.11.2 for Flink 1.11.
> >
> > Most noticeable fixes are:
> > - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> > - FLINK-18902 Cannot serve results of asynchronous REST operations in
> > per-job mode
> > - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> > - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> > - FLINK-18646 Managed memory released check can block RPC thread
> > - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> > JobManagerFlinkMemoryUtils.java
> > - FLINK-18663 RestServerEndpoint may prevent server shutdown
> > - FLINK-18595 Deadlock during job shutdown
> > - FLINK-18581 Cannot find GC cleaner with java version previous
> > jdk8u72(-b01)
> > - FLINK-17075 Add task status reconciliation between TM and JM
> >
> > Furthermore, I think the following blocker issue should be merged before
> > 1.11.2 release
> >
> > - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
> >
> > I would volunteer as the release manager and kick off the release
> process.
> > What do you think?
> >
> > Please let me know if there are any concerns or any missing blocker
> issues
> > need to be fixed in 1.11.2.
> >
> > Thanks,
> > Zhu Zhu
> >
>
Re: [DISCUSS] Releasing Flink 1.11.2
Posted by David Anderson <da...@alpinegizmo.com>.
I think it's worth considering whether we can get this bugfix included in
1.11.2:
- FLINK-19109 Split Reader eats chained periodic watermarks
There is a PR, but it's still a work in progress. Cc'ing Roman, who has
been working on this.
Regards,
David
On Wed, Sep 2, 2020 at 2:19 PM Zhu Zhu <re...@gmail.com> wrote:
> Hi All,
>
> It has been about 1 month since we released Flink 1.11.1. It's not too far
> but
> we already have more than 80 resolved improvements/bugs in the release-1.11
> branch. Some of them are quite critical. Therefore, I propose to create the
> next
> bugfix release 1.11.2 for Flink 1.11.
>
> Most noticeable fixes are:
> - FLINK-18769 MiniBatch doesn't work with FLIP-95 source
> - FLINK-18902 Cannot serve results of asynchronous REST operations in
> per-job mode
> - FLINK-18682 Vector orc reader cannot read Hive 2.0.0 table
> - FLINK-18608 CustomizedConvertRule#convertCast drops nullability
> - FLINK-18646 Managed memory released check can block RPC thread
> - FLINK-18993 Invoke sanityCheckTotalFlinkMemory method incorrectly in
> JobManagerFlinkMemoryUtils.java
> - FLINK-18663 RestServerEndpoint may prevent server shutdown
> - FLINK-18595 Deadlock during job shutdown
> - FLINK-18581 Cannot find GC cleaner with java version previous
> jdk8u72(-b01)
> - FLINK-17075 Add task status reconciliation between TM and JM
>
> Furthermore, I think the following blocker issue should be merged before
> 1.11.2 release
>
> - FLINK-19121 Avoid accessing HDFS frequently in HiveBulkWriterFactory
>
> I would volunteer as the release manager and kick off the release process.
> What do you think?
>
> Please let me know if there are any concerns or any missing blocker issues
> need to be fixed in 1.11.2.
>
> Thanks,
> Zhu Zhu
>