You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Yu Li <ca...@gmail.com> on 2020/03/10 09:49:09 UTC
[DISCUSS] Releasing Flink 1.10.1
Hi All,
It has been almost one month since we released Flink 1.10.0. We already
have more than 40 resolved improvements/bugs in the release-1.10 branch,
and I propose to start the 1.10.1 release cycle.
Most noticeable fixes are:
- FLINK-16241 [legal] Remove the license and notice file in flink-ml-lib
module
- FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
- FLINK-16161 Statistics zero should be known in HiveCatalog
- FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when mapping
- FLINK-16108 StreamSQLExample is failed if running in blink planner
- FLINK-16139 Co-location constraints are not reset on task recovery in
DefaultScheduler
- FLINK-16414 Create udaf/udtf function using sql casuing
ValidationException: SQL validation failed
Furthermore, I think the following issues should be merged before 1.10.1
release (especially the Metaspace OOM issue):
- FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
submission
- FLINK-16406 Increase default value for JVM Metaspace to minimise its
OutOfMemoryError
- FLINK-16047 Blink planner produces wrong aggregate results with state
clean up
- FLINK-16070 Blink planner can not extract correct unique key for
UpsertStreamTableSink
I would volunteer as the release manager and kick off the release process
once blocker issues are merged. What do you think?
If there are any concerns or missing blocker issues need to be fixed in
1.10.1, please let me know. Thanks.
Best Regards,
Yu
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
FLINK-16662 has been fixed just now (thanks @LionelZ and @Klou for the
efforts!) and I will prepare the first RC soon, JFYI.
Best Regards,
Yu
On Thu, 16 Apr 2020 at 23:29, Yu Li <ca...@gmail.com> wrote:
> Thanks for the efforts and feedbacks all.
>
> Now the only left blocker is the below one (newly escalated with a solid
> reason), will create RC1 right after it's resolved.
>
> FLINK-16662 Blink Planner failed to generate JobGraph for POJO DataStream
> converting to Table (Cannot determine simple type name)
>
> Best Regards,
> Yu
>
>
> On Thu, 16 Apr 2020 at 20:17, Congxian Qiu <qc...@gmail.com> wrote:
>
>> Thanks a lot for driving this, Yu! Looking forward to the first RC of
>> 1.10.1
>> FLINK-16576 has been merged into release-1.10 already.
>>
>> Best,
>> Congxian
>>
>>
>> Dian Fu <di...@gmail.com> 于2020年4月16日周四 上午10:35写道：
>>
>> > Thanks a lot for driving this, Yu! Looking forward for the first RC of
>> > 1.10.1.
>> >
>> > > 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
>> > >
>> > > Looking forward the first RC of Flink 1.10.1 .
>> > > Good job Yu!
>> > >
>> > > Best,
>> > > Jincheng
>> > >
>> > >
>> > >
>> > > Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
>> > >
>> > >> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
>> > >> released.
>> > >>
>> > >> Best,
>> > >> Jark
>> > >>
>> > >> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org>
>> > wrote:
>> > >>
>> > >>> Great to see that will have the first RC for Flink 1.10.1 soon.
>> Thanks
>> > a
>> > >>> lot for driving this effort Yu!
>> > >>>
>> > >>> Cheers,
>> > >>> Till
>> > >>>
>> > >>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
>> > >>>
>> > >>>> Thanks Weike and all others for the efforts!
>> > >>>>
>> > >>>> Here comes the latest status, we are in good shape and plan to
>> produce
>> > >>> RC1
>> > >>>> next week.
>> > >>>>
>> > >>>> * Blockers (1 left)
>> > >>>> - [Closed] FLINK-16018 Improve error reporting when submitting
>> batch
>> > >>> job
>> > >>>> (instead of AskTimeoutException)
>> > >>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>> > >>> repeated
>> > >>>> job submission
>> > >>>> - [Closed] FLINK-16170 SearchTemplateRequest
>> ClassNotFoundException
>> > >>> when
>> > >>>> use flink-sql-connector-elasticsearch7
>> > >>>> - [Closed] FLINK-16262 Class loader problem with
>> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > >>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> > >>>> minimise its OutOfMemoryError
>> > >>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>> > >>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
>> > >>> client
>> > >>>> can retrieve JobResult
>> > >>>> - [Closed] FLINK-16913
>> ReadableConfigToConfigurationAdapter#getEnum
>> > >>>> throws UnsupportedOperationException
>> > >>>> - [Closed] FLINK-16626 Exception encountered when cancelling a
>> job in
>> > >>>> yarn per-job mode
>> > >>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work
>> when
>> > >> the
>> > >>>> input column is of composite type
>> > >>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
>> > >> memory
>> > >>>> state backends
>> > >>>>
>> > >>>> * Critical (1 left)
>> > >>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>> results
>> > >>>> with state clean up
>> > >>>> - [Closed] FLINK-16070 Blink planner can not extract correct
>> unique
>> > >> key
>> > >>>> for UpsertStreamTableSink
>> > >>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
>> should
>> > >>> be
>> > >>>> handled as Fatal Error in TaskManager
>> > >>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
>> > >> error
>> > >>>> - [May Postpone] FLINK-16408 Bind user code class loader to
>> lifetime
>> > >>> of a
>> > >>>> slot
>> > >>>>
>> > >>>> Please let me know if any concerns/comments. Thanks.
>> > >>>>
>> > >>>> Best Regards,
>> > >>>> Yu
>> > >>>>
>> > >>>>
>> > >>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
>> > >>> wrote:
>> > >>>>
>> > >>>>> Hi Yu,
>> > >>>>>
>> > >>>>> Thanks for your updates. I am still working on the fix for
>> > >> FLINK-16626
>> > >>>> and
>> > >>>>> it is expected to be completed by this Sunday after thorough
>> testing.
>> > >>>>>
>> > >>>>> Sincerely,
>> > >>>>> Weike
>> > >>>>>
>> > >>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
>> > >>>>>
>> > >>>>>> Updates for 1.10.1 watched issues (we are in good progress and
>> > >> almost
>> > >>>>>> there
>> > >>>>>> to produce the first RC, thanks all for the efforts):
>> > >>>>>>
>> > >>>>>> * Blockers (3 left)
>> > >>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
>> > >> batch
>> > >>>> job
>> > >>>>>> (instead of AskTimeoutException)
>> > >>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>> > >>>>>> repeated
>> > >>>>>> job submission
>> > >>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
>> > >> ClassNotFoundException
>> > >>>> when
>> > >>>>>> use flink-sql-connector-elasticsearch7
>> > >>>>>> - [Closed] FLINK-16262 Class loader problem with
>> > >>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > >>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
>> to
>> > >>>>>> minimise its OutOfMemoryError
>> > >>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>> > >>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
>> > >>> memory
>> > >>>>>> state backends
>> > >>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
>> > >>>> cancelling a
>> > >>>>>> job in yarn per-job mode
>> > >>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster
>> before
>> > >>>>>> client
>> > >>>>>> can retrieve JobResult
>> > >>>>>> - [PR reviewed] FLINK-16913
>> > >>>> ReadableConfigToConfigurationAdapter#getEnum
>> > >>>>>> throws UnsupportedOperationException
>> > >>>>>>
>> > >>>>>> * Critical (1 left)
>> > >>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
>> > >>> error
>> > >>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>> > >>> results
>> > >>>>>> with state clean up
>> > >>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
>> > >> unique
>> > >>>> key
>> > >>>>>> for UpsertStreamTableSink
>> > >>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
>> > >>> should
>> > >>>> be
>> > >>>>>> handled as Fatal Error in TaskManager
>> > >>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
>> > >> lifetime
>> > >>>> of
>> > >>>>>> a
>> > >>>>>> slot
>> > >>>>>>
>> > >>>>>> Please let me know if you find any missing ones, thanks.
>> > >>>>>>
>> > >>>>>> Best Regards,
>> > >>>>>> Yu
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
>> > >>>>>>
>> > >>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
>> > >>>>>>>
>> > >>>>>>> * Blockers (4 left)
>> > >>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
>> > >>>>>> submitting
>> > >>>>>>> batch job (instead of AskTimeoutException)
>> > >>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error
>> on
>> > >>>>>>> repeated job submission
>> > >>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
>> > >>> ClassNotFoundException
>> > >>>>>> when
>> > >>>>>>> use flink-sql-connector-elasticsearch7
>> > >>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
>> > >>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > >>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
>> > >> to
>> > >>>>>>> minimise its OutOfMemoryError
>> > >>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
>> files
>> > >>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
>> > >>>> memory
>> > >>>>>>> state backends
>> > >>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
>> > >>>>>> MiniCluster
>> > >>>>>>> before client can retrieve JobResult
>> > >>>>>>>
>> > >>>>>>> * Critical (2 left)
>> > >>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>> > >>>> results
>> > >>>>>>> with state clean up
>> > >>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
>> > >> correct
>> > >>>>>>> unique key for UpsertStreamTableSink
>> > >>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
>> > >> should
>> > >>>> be
>> > >>>>>>> handled as Fatal Error in TaskManager
>> > >>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
>> > >> a
>> > >>>> slot
>> > >>>>>>>
>> > >>>>>>> There's a new blocker added. Please let me know if you find any
>> > >>> other
>> > >>>>>>> issues should be put into the watch list. Thanks.
>> > >>>>>>>
>> > >>>>>>> Best Regards,
>> > >>>>>>> Yu
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
>> > >>>>>>>
>> > >>>>>>>> Hi All,
>> > >>>>>>>>
>> > >>>>>>>> Here is the status update of issues in 1.10.1 watch list:
>> > >>>>>>>>
>> > >>>>>>>> * Blockers (7)
>> > >>>>>>>>
>> > >>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
>> > >>>>>>>> submitting batch job (instead of AskTimeoutException)
>> > >>>>>>>>
>> > >>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
>> > >> OOM
>> > >>>>>> error
>> > >>>>>>>> on repeated job submission
>> > >>>>>>>>
>> > >>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> > >>>>>>>> ClassNotFoundException when use
>> > >> flink-sql-connector-elasticsearch7
>> > >>>>>>>>
>> > >>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
>> > >>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > >>>>>>>>
>> > >>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
>> Metaspace
>> > >>> to
>> > >>>>>>>> minimise its OutOfMemoryError
>> > >>>>>>>>
>> > >>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
>> > >> files
>> > >>>>>>>>
>> > >>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
>> memory
>> > >>>> state
>> > >>>>>>>> backends
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> * Critical (4)
>> > >>>>>>>>
>> > >>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>> > >>>> results
>> > >>>>>>>> with state clean up
>> > >>>>>>>>
>> > >>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
>> > >>> correct
>> > >>>>>>>> unique key for UpsertStreamTableSink
>> > >>>>>>>>
>> > >>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
>> should
>> > >>> be
>> > >>>>>>>> handled as Fatal Error in TaskManager
>> > >>>>>>>>
>> > >>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
>> > >> of a
>> > >>>>>> slot
>> > >>>>>>>>
>> > >>>>>>>> Please let me know if you see any new blockers to add to the
>> > >> list.
>> > >>>>>> Thanks.
>> > >>>>>>>>
>> > >>>>>>>> Best Regards,
>> > >>>>>>>> Yu
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Thanks for the updates Till!
>> > >>>>>>>>>
>> > >>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
>> > >> and
>> > >>>>>>>>> complete fix separately, and only include the easy one in
>> > >> 1.10.1?
>> > >>> Or
>> > >>>>>> please
>> > >>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
>> > >>>>>> nothing"
>> > >>>>>>>>> policy is preferred (smile). Thanks.
>> > >>>>>>>>>
>> > >>>>>>>>> Best Regards,
>> > >>>>>>>>> Yu
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
>> > >> trohrmann@apache.org
>> > >>>>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
>> > >> our
>> > >>>>>> release
>> > >>>>>>>>>> manager Yu.
>> > >>>>>>>>>>
>> > >>>>>>>>>> I think we can soon merge the increase of metaspace size and
>> > >>>>>> improving
>> > >>>>>>>>>> the
>> > >>>>>>>>>> error message. The assumption is that we currently don't have
>> > >> too
>> > >>>>>> many
>> > >>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
>> > >>> course,
>> > >>>>>> the
>> > >>>>>>>>>> sooner we release the bug fix release, the fewer deployments
>> > >> will
>> > >>>> be
>> > >>>>>>>>>> affected by this change.
>> > >>>>>>>>>>
>> > >>>>>>>>>> For FLINK-16018, I think there would be an easy solution
>> which
>> > >> we
>> > >>>>>> could
>> > >>>>>>>>>> include in the bug fix release. The proper fix will most
>> likely
>> > >>>> take
>> > >>>>>> a
>> > >>>>>>>>>> bit
>> > >>>>>>>>>> longer, though.
>> > >>>>>>>>>>
>> > >>>>>>>>>> Cheers,
>> > >>>>>>>>>> Till
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
>> > >>>>>> azagrebin@apache.org>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
>> > >> user
>> > >>>>>>>>>> mailing
>> > >>>>>>>>>>> list
>> > >>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
>> > >>>> Specifically
>> > >>>>>>>>>> asking
>> > >>>>>>>>>>>> for who has very small TM setups?
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> There has been a survey about this topic since 10 days:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
>> > >> 1.10`
>> > >>>>>>>>>>> I can ask there about the small TM setups.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
>> > >>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
>> > >>> on
>> > >>>>>>>>>> restore
>> > >>>>>>>>>>> with
>> > >>>>>>>>>>>> memory state backends
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
>> > >>>>>> There're
>> > >>>>>>>>>>> totally
>> > >>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
>> > >>>>>> which 14
>> > >>>>>>>>>>>> already resolved and 31 left, and the below ones are
>> > >> watched
>> > >>>>>>>>>> (meaning
>> > >>>>>>>>>>> that
>> > >>>>>>>>>>>> once the below ones got fixed, we will kick of the
>> > >> releasing
>> > >>>> with
>> > >>>>>>>>>> left
>> > >>>>>>>>>>> ones
>> > >>>>>>>>>>>> postponed to 1.10.2 unless objections):
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> * Blockers (7)
>> > >>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
>> > >>> when
>> > >>>>>>>>>>> submitting
>> > >>>>>>>>>>>> batch job (instead of AskTimeoutException)
>> > >>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
>> > >>> Metaspace
>> > >>>>>> OOM
>> > >>>>>>>>>> error
>> > >>>>>>>>>>>> on repeated job submission
>> > >>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> > >>>>>>>>>>> ClassNotFoundException
>> > >>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
>> > >>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
>> > >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
>> > >> directory
>> > >>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
>> > >>>>>> Metaspace to
>> > >>>>>>>>>>>> minimise its OutOfMemoryError
>> > >>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
>> > >>>> files
>> > >>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
>> > >>>> memory
>> > >>>>>>>>>> state
>> > >>>>>>>>>>>> backends
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> * Critical (4)
>> > >>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
>> > >> aggregate
>> > >>>>>>>>>> results
>> > >>>>>>>>>>> with
>> > >>>>>>>>>>>> state clean up
>> > >>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
>> > >> extract
>> > >>>>>> correct
>> > >>>>>>>>>>> unique
>> > >>>>>>>>>>>> key for UpsertStreamTableSink
>> > >>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
>> > >>>>>> should be
>> > >>>>>>>>>>>> handled as Fatal Error in TaskManager
>> > >>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
>> > >>> lifetime
>> > >>>>>> of a
>> > >>>>>>>>>> slot
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Please raise your hand if you find any other issues should
>> > >> be
>> > >>>> put
>> > >>>>>>>>>> into
>> > >>>>>>>>>>> this
>> > >>>>>>>>>>>> list, thanks.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Please also note that 1.10.2 version is already created
>> > >>> (thanks
>> > >>>>>> for
>> > >>>>>>>>>> the
>> > >>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
>> > >>> to
>> > >>>> it
>> > >>>>>>>>>> when
>> > >>>>>>>>>>>> necessary.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
>> > >> sewen@apache.org
>> > >>>>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
>> > >>> user
>> > >>>>>>>>>> mailing
>> > >>>>>>>>>>>> list
>> > >>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
>> > >>>>>> Specifically
>> > >>>>>>>>>>> asking
>> > >>>>>>>>>>>>> for who has very small TM setups?
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
>> > >>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Thanks for the reminder Stephan and the
>> > >> inputs/discussion
>> > >>>>>>>>>> Andrey and
>> > >>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
>> > >>>>>>>>>> introduce new
>> > >>>>>>>>>>>>>> issues when resolving the existing one. Will watch
>> > >>> closely
>> > >>>>>>>>>> about the
>> > >>>>>>>>>>>>>> discussion and conclusion around metaspace
>> > >> configuration.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
>> > >>> optimistic
>> > >>>>>> that
>> > >>>>>>>>>>>>> reserving
>> > >>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
>> > >>> cause a
>> > >>>>>> big
>> > >>>>>>>>>>>> problem,
>> > >>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
>> > >>>>>> memory
>> > >>>>>>>>>> per TM
>> > >>>>>>>>>>>> is
>> > >>>>>>>>>>>>>> the common case.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
>> > >>>>>>>>>> tonysong820@gmail.com>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
>> > >>>>>> default
>> > >>>>>>>>>> value
>> > >>>>>>>>>>>> for
>> > >>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
>> > >>>> default
>> > >>>>>>>>>>>>>>> 'flink-conf.yaml'.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> That means:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
>> > >>> it's
>> > >>>>>> not
>> > >>>>>>>>>>> really
>> > >>>>>>>>>>>>>> taking
>> > >>>>>>>>>>>>>>> memory from heap / managed for metaspace.
>> > >>>>>>>>>>>>>>> - For people using custom configuration files:
>> > >>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
>> > >>>> metaspace
>> > >>>>>>>>>> does
>> > >>>>>>>>>>> not
>> > >>>>>>>>>>>>>> really
>> > >>>>>>>>>>>>>>> affect the flink memory internal components
>> > >>> (heap,
>> > >>>>>>>>>> managed,
>> > >>>>>>>>>>>>> etc.).
>> > >>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
>> > >>>> likely
>> > >>>>>>>>>> active
>> > >>>>>>>>>>>>>> setups),
>> > >>>>>>>>>>>>>>> then they will have less heap / managed memory,
>> > >>> or
>> > >>>>>>>>>> worse,
>> > >>>>>>>>>>> their
>> > >>>>>>>>>>>>>>> configuration may not work without adjustment.
>> > >>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
>> > >> 4
>> > >>> GB
>> > >>>>>> or
>> > >>>>>>>>>> more),
>> > >>>>>>>>>>>> it
>> > >>>>>>>>>>>>> is
>> > >>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
>> > >>> more
>> > >>>>>>>>>> memory.
>> > >>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
>> > >>> GB)
>> > >>>>>> the
>> > >>>>>>>>>>>> metaspace
>> > >>>>>>>>>>>>>>> increase is more likely to cause problem.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> So basically only people have small 'process.size' in
>> > >>>>>> custom
>> > >>>>>>>>>> config
>> > >>>>>>>>>>>>> file
>> > >>>>>>>>>>>>>>> are really affected. I'm not sure what is the
>> > >>> proportion
>> > >>>> of
>> > >>>>>>>>>> such
>> > >>>>>>>>>>> use
>> > >>>>>>>>>>>>>> cases
>> > >>>>>>>>>>>>>>> though. (From questions asked on the user ML,
>> > >> probably
>> > >>>> not
>> > >>>>>>>>>> much).
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Thank you~
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Xintong Song
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
>> > >>>>>>>>>> sewen@apache.org>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
>> > >> should
>> > >>>> not
>> > >>>>>> go
>> > >>>>>>>>>> into
>> > >>>>>>>>>>>>>> 1.10.1,
>> > >>>>>>>>>>>>>>> I
>> > >>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> We have to trade off the fact that we may break
>> > >> some
>> > >>>>>>>>>> deployments
>> > >>>>>>>>>>>> with
>> > >>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
>> > >>>>>>>>>>>>>>>> I simply lack the data points / insight at the
>> > >> moment
>> > >>>> to
>> > >>>>>>>>>>> understand
>> > >>>>>>>>>>>>> how
>> > >>>>>>>>>>>>>>>> significant both cases are, meaning how many users
>> > >>>> would
>> > >>>>>> be
>> > >>>>>>>>>>>> affected
>> > >>>>>>>>>>>>>> and
>> > >>>>>>>>>>>>>>>> how badly.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Independent of that, improving the error message is
>> > >>>>>> always
>> > >>>>>>>>>>> helpful.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>> > >>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
>> > >>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
>> > >> because
>> > >>>>>> users
>> > >>>>>>>>>> expect
>> > >>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> upgrade
>> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
>> > >>> be
>> > >>>>>>>>>> possible
>> > >>>>>>>>>>>> with
>> > >>>>>>>>>>>>>> that
>> > >>>>>>>>>>>>>>>>>> change.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
>> > >> only
>> > >>>>>> try to
>> > >>>>>>>>>>>> improve
>> > >>>>>>>>>>>>>>> error
>> > >>>>>>>>>>>>>>>>> message and docs.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
>> > >>>>>>>>>> sewen@apache.org>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
>> > >>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
>> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
>> > >> because
>> > >>>>>> users
>> > >>>>>>>>>> expect
>> > >>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> upgrade
>> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
>> > >>> be
>> > >>>>>>>>>> possible
>> > >>>>>>>>>>>> with
>> > >>>>>>>>>>>>>> that
>> > >>>>>>>>>>>>>>>>>> change.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
>> > >> Zagrebin
>> > >>> <
>> > >>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
>> > >>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
>> > >> Metaspace
>> > >>>> OOM
>> > >>>>>>>>>> error
>> > >>>>>>>>>>> on
>> > >>>>>>>>>>>>>>> repeated
>> > >>>>>>>>>>>>>>>>>>> job”
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
>> > >>>> covered
>> > >>>>>>>>>> by:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
>> > >>> be
>> > >>>>>>>>>> handled as
>> > >>>>>>>>>>>>> Fatal
>> > >>>>>>>>>>>>>>>> Error
>> > >>>>>>>>>>>>>>>>>>> in TaskManager
>> > >>>>>>>>>>>>>>>>>>> no full consensus there but improving error
>> > >>>>>> message
>> > >>>>>>>>>> for
>> > >>>>>>>>>>>>> existing
>> > >>>>>>>>>>>>>>>> task
>> > >>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
>> > >>>>>>>>>> Metaspace to
>> > >>>>>>>>>>>>>> minimise
>> > >>>>>>>>>>>>>>>> its
>> > >>>>>>>>>>>>>>>>>>> OutOfMemoryError
>> > >>>>>>>>>>>>>>>>>>> see further
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
>> > >>> "SdkMBeanRegistrySupport"
>> > >>>>>> from
>> > >>>>>>>>>>>>> dynamically
>> > >>>>>>>>>>>>>>>>> loaded
>> > >>>>>>>>>>>>>>>>>>> AWS connectors
>> > >>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
>> > >>>> close
>> > >>>>>> to
>> > >>>>>>>>>> be
>> > >>>>>>>>>>>>> resolved
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
>> > >>> JVM
>> > >>>>>>>>>>> Metaspace"
>> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
>> > >>>> bugfix
>> > >>>>>>>>>> release?
>> > >>>>>>>>>>>> It
>> > >>>>>>>>>>>>>>>> changes
>> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
>> > >>>>>> memory
>> > >>>>>>>>>> on
>> > >>>>>>>>>>>>> existing
>> > >>>>>>>>>>>>>>>> setups
>> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
>> > >>>>>> resolved
>> > >>>>>>>>>> the
>> > >>>>>>>>>>>>> reported
>> > >>>>>>>>>>>>>>>>> problems
>> > >>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
>> > >>> merged
>> > >>>>>> it
>> > >>>>>>>>>> today
>> > >>>>>>>>>>> as
>> > >>>>>>>>>>>>>> there
>> > >>>>>>>>>>>>>>>>> were
>> > >>>>>>>>>>>>>>>>>>> no more concerns.
>> > >>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
>> > >> message
>> > >>>>>> error
>> > >>>>>>>>>> with
>> > >>>>>>>>>>>>>>> reference
>> > >>>>>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>>>>> the metaspace option should help the most
>> > >>>>>>>>>>>>>>>>>>> because user would not have to read all docs
>> > >> to
>> > >>>> fix
>> > >>>>>> it
>> > >>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>> Andrey
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>> > >>>>>>>>>> sewen@apache.org>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
>> > >> Metaspace
>> > >>>> OOM
>> > >>>>>>>>>> error
>> > >>>>>>>>>>> on
>> > >>>>>>>>>>>>>>> repeated
>> > >>>>>>>>>>>>>>>>>>> job"
>> > >>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
>> > >> exact
>> > >>>>>>>>>> solution,
>> > >>>>>>>>>>> yet,
>> > >>>>>>>>>>>>> and
>> > >>>>>>>>>>>>>>>> some
>> > >>>>>>>>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
>> > >>> are
>> > >>>>>>>>>> hard to
>> > >>>>>>>>>>>>>> predict,
>> > >>>>>>>>>>>>>>>> so I
>> > >>>>>>>>>>>>>>>>>>> am
>> > >>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
>> > >>> JVM
>> > >>>>>>>>>>> Metaspace"
>> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
>> > >>>> bugfix
>> > >>>>>>>>>> release?
>> > >>>>>>>>>>>> It
>> > >>>>>>>>>>>>>>>> changes
>> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
>> > >>>>>> memory
>> > >>>>>>>>>> on
>> > >>>>>>>>>>>>> existing
>> > >>>>>>>>>>>>>>>> setups
>> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
>> > >>> two
>> > >>>>>>>>>> issues
>> > >>>>>>>>>>> and
>> > >>>>>>>>>>>>>> think
>> > >>>>>>>>>>>>>>>>> about
>> > >>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
>> > >>>>>>>>>> carp84@gmail.com>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
>> > >> eye
>> > >>>> on
>> > >>>>>>>>>> these
>> > >>>>>>>>>>> two.
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>> > >>>>>>>>>> imjark@gmail.com>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
>> > >>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
>> > >> should
>> > >>>> also
>> > >>>>>>>>>> try to
>> > >>>>>>>>>>>> get
>> > >>>>>>>>>>>>> in
>> > >>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>>>> following issues:
>> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
>> > >>>> flink-conf
>> > >>>>>>>>>>>> parameters
>> > >>>>>>>>>>>>>> from
>> > >>>>>>>>>>>>>>>> SQL
>> > >>>>>>>>>>>>>>>>>>> CLI
>> > >>>>>>>>>>>>>>>>>>>>>> environment
>> > >>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
>> > >>>>>>>>>> watermark
>> > >>>>>>>>>>>>> interval,
>> > >>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
>> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
>> > >>>>>> uncatched
>> > >>>>>>>>>>>>> exception
>> > >>>>>>>>>>>>>> is
>> > >>>>>>>>>>>>>>>>>>> thrown
>> > >>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
>> > >> using
>> > >>>> SQL
>> > >>>>>> CLI
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>> Jark
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
>> > >>>>>>>>>> carp84@gmail.com>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
>> > >> added
>> > >>>>>>>>>> 1.10.1 into
>> > >>>>>>>>>>>>>>>> FLINK-16225
>> > >>>>>>>>>>>>>>>>>>>>> fix
>> > >>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
>> > >>>> Critical.
>> > >>>>>>>>>> Will
>> > >>>>>>>>>>> also
>> > >>>>>>>>>>>>>> watch
>> > >>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
>> > >>> Zagrebin
>> > >>>> <
>> > >>>>>>>>>>>>>>>> azagrebin@apache.org
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
>> > >>>>>>>>>> discussion!
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> Apart from
>> > >>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
>> > >>> JVM
>> > >>>>>>>>>> Metaspace
>> > >>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> minimise
>> > >>>>>>>>>>>>>>>>>>>>> its
>> > >>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
>> > >>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
>> > >>>>>> following
>> > >>>>>>>>>>>> issues:
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
>> > >>>> should
>> > >>>>>> be
>> > >>>>>>>>>>> handled
>> > >>>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>> Fatal
>> > >>>>>>>>>>>>>>>>>>>>>> Error
>> > >>>>>>>>>>>>>>>>>>>>>>> in
>> > >>>>>>>>>>>>>>>>>>>>>>>> TaskManager
>> > >>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
>> > >>> even
>> > >>>>>> in a
>> > >>>>>>>>>>> better
>> > >>>>>>>>>>>>> way
>> > >>>>>>>>>>>>>>>>> because
>> > >>>>>>>>>>>>>>>>>>>>>> OOM
>> > >>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
>> > >>>>>> immediately
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
>> > >> loader
>> > >>>> to
>> > >>>>>>>>>> lifetime
>> > >>>>>>>>>>>> of
>> > >>>>>>>>>>>>> a
>> > >>>>>>>>>>>>>>> slot
>> > >>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
>> > >>> against
>> > >>>>>> class
>> > >>>>>>>>>>>> loading
>> > >>>>>>>>>>>>>>> leaks
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
>> > >>> when
>> > >>>>>>>>>> submitting
>> > >>>>>>>>>>>>> batch
>> > >>>>>>>>>>>>>>> job
>> > >>>>>>>>>>>>>>>>>>>>>>> (instead
>> > >>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
>> > >>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
>> > >>>> multiple
>> > >>>>>>>>>> users
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>>>> Andrey
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
>> > >>> Li <
>> > >>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
>> > >>> the
>> > >>>>>>>>>> 1.10.1
>> > >>>>>>>>>>>>> release.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
>> > >> are
>> > >>>>>> looking
>> > >>>>>>>>>>>> forward
>> > >>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>> them.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
>> > >>> Guo <
>> > >>>>>>>>>>>>>>> karmagyz@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
>> > >> cycle.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
>> > >>>> Song
>> > >>>>>> <
>> > >>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
>> > >>> get
>> > >>>>>> some
>> > >>>>>>>>>>> inputs
>> > >>>>>>>>>>>>> for
>> > >>>>>>>>>>>>>>>>>>>>>>>> prioritizing
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
>> > >>> good
>> > >>>> to
>> > >>>>>>>>>> me.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
>> > >> Li <
>> > >>>>>>>>>>>>> carp84@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
>> > >>> patch
>> > >>>>>>>>>> release
>> > >>>>>>>>>>> of
>> > >>>>>>>>>>>> a
>> > >>>>>>>>>>>>>>> major
>> > >>>>>>>>>>>>>>>>>>>>>>>> version
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> will
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
>> > >>>>>>>>>> depending on
>> > >>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>> speed
>> > >>>>>>>>>>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>>>>>>>>>>>>> blocker
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
>> > >> 28th
>> > >>>>>> [1],
>> > >>>>>>>>>>> released
>> > >>>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>> Jul
>> > >>>>>>>>>>>>>>>>>>>>>> 3rd
>> > >>>>>>>>>>>>>>>>>>>>>>>> [2]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
>> > >> 23rd
>> > >>>>>> [3],
>> > >>>>>>>>>>> released
>> > >>>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>> Oct
>> > >>>>>>>>>>>>>>>>>>>>>> 19th
>> > >>>>>>>>>>>>>>>>>>>>>>>> [4]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
>> > >>>>>> course,
>> > >>>>>>>>>> but
>> > >>>>>>>>>>>> could
>> > >>>>>>>>>>>>>> use
>> > >>>>>>>>>>>>>>>>>>>>> it
>> > >>>>>>>>>>>>>>>>>>>>>>> as a
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
>> > >> let
>> > >>> me
>> > >>>>>>>>>> know if
>> > >>>>>>>>>>> any
>> > >>>>>>>>>>>>>>>>>>>>> concerns
>> > >>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>
>> > >>
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
>> > >>>> Song
>> > >>>>>> <
>> > >>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
>> > >>>>>> volunteering
>> > >>>>>>>>>> to be
>> > >>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>>>> release
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> manager.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
>> > >>>> plan
>> > >>>>>> for
>> > >>>>>>>>>>>> 1.10.1?
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
>> > >>>> Zhijiang
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
>> > >> .invalid>
>> > >>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
>> > >> Yu!
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>> ------------------------------------------------------------------
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
>> > >> Flink
>> > >>>>>> 1.10.1
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
>> > >> Yes
>> > >>>> will
>> > >>>>>>>>>> also
>> > >>>>>>>>>>> keep
>> > >>>>>>>>>>>>> an
>> > >>>>>>>>>>>>>>>>>>>>> eye
>> > >>>>>>>>>>>>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>>>>>>>>>>>> these
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
>> > >>>>>> Cheng <
>> > >>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
>> > >>>> discussion
>> > >>>>>> and
>> > >>>>>>>>>>>>> volunteer
>> > >>>>>>>>>>>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> release
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
>> > >>> issues[1]
>> > >>>>>>>>>> which are
>> > >>>>>>>>>>>>>> marked
>> > >>>>>>>>>>>>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>>>>>>>>> a
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
>> > >>>> year
>> > >>>>>> in
>> > >>>>>>>>>>> NOTICE
>> > >>>>>>>>>>>>>> files
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
>> > >>>> with
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
>> > >>>>>>>>>> and
>> > >>>>>>>>>>>> usrlib
>> > >>>>>>>>>>>>>>>>>>>>>>> directory
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
>> > >> SearchTemplateRequest
>> > >>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
>> > >>>>>>>>>>>>>>>>>>>>>>>> when
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> use
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
>> > >>> reporting
>> > >>>>>> when
>> > >>>>>>>>>>>>> submitting
>> > >>>>>>>>>>>>>>>>>>>>>> batch
>> > >>>>>>>>>>>>>>>>>>>>>>>> job
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
>> > >>> in
>> > >>>>>>>>>> 1.10.1.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>
>> > >> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
>> > >>> Li
>> > >>>> <
>> > >>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
>> > >>> helpful.
>> > >>>>>>>>>> Thanks a
>> > >>>>>>>>>>>> lot!
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
>> > >>>> jincheng
>> > >>>>>>>>>> sun <
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
>> > >> discussion
>> > >>>>>> Yu. I
>> > >>>>>>>>>> would
>> > >>>>>>>>>>>>> like
>> > >>>>>>>>>>>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>>>>>>>>>>> give
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> you a
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
>> > >>>>>>>>>> finished.(If you
>> > >>>>>>>>>>>>>>>>>>>>> need)
>> > >>>>>>>>>>>>>>>>>>>>>>> :)
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>
>>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the efforts and feedbacks all.
Now the only left blocker is the below one (newly escalated with a solid
reason), will create RC1 right after it's resolved.
FLINK-16662 Blink Planner failed to generate JobGraph for POJO DataStream
converting to Table (Cannot determine simple type name)
Best Regards,
Yu
On Thu, 16 Apr 2020 at 20:17, Congxian Qiu <qc...@gmail.com> wrote:
> Thanks a lot for driving this, Yu! Looking forward to the first RC of
> 1.10.1
> FLINK-16576 has been merged into release-1.10 already.
>
> Best,
> Congxian
>
>
> Dian Fu <di...@gmail.com> 于2020年4月16日周四 上午10:35写道：
>
> > Thanks a lot for driving this, Yu! Looking forward for the first RC of
> > 1.10.1.
> >
> > > 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
> > >
> > > Looking forward the first RC of Flink 1.10.1 .
> > > Good job Yu!
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > >
> > > Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
> > >
> > >> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> > >> released.
> > >>
> > >> Best,
> > >> Jark
> > >>
> > >> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org>
> > wrote:
> > >>
> > >>> Great to see that will have the first RC for Flink 1.10.1 soon.
> Thanks
> > a
> > >>> lot for driving this effort Yu!
> > >>>
> > >>> Cheers,
> > >>> Till
> > >>>
> > >>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> > >>>
> > >>>> Thanks Weike and all others for the efforts!
> > >>>>
> > >>>> Here comes the latest status, we are in good shape and plan to
> produce
> > >>> RC1
> > >>>> next week.
> > >>>>
> > >>>> * Blockers (1 left)
> > >>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> batch
> > >>> job
> > >>>> (instead of AskTimeoutException)
> > >>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>> repeated
> > >>>> job submission
> > >>>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> > >>> when
> > >>>> use flink-sql-connector-elasticsearch7
> > >>>> - [Closed] FLINK-16262 Class loader problem with
> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > >>>> minimise its OutOfMemoryError
> > >>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> > >>> client
> > >>>> can retrieve JobResult
> > >>>> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> > >>>> throws UnsupportedOperationException
> > >>>> - [Closed] FLINK-16626 Exception encountered when cancelling a job
> in
> > >>>> yarn per-job mode
> > >>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
> > >> the
> > >>>> input column is of composite type
> > >>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> > >> memory
> > >>>> state backends
> > >>>>
> > >>>> * Critical (1 left)
> > >>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> results
> > >>>> with state clean up
> > >>>> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> > >> key
> > >>>> for UpsertStreamTableSink
> > >>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> should
> > >>> be
> > >>>> handled as Fatal Error in TaskManager
> > >>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> > >> error
> > >>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> lifetime
> > >>> of a
> > >>>> slot
> > >>>>
> > >>>> Please let me know if any concerns/comments. Thanks.
> > >>>>
> > >>>> Best Regards,
> > >>>> Yu
> > >>>>
> > >>>>
> > >>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> > >>> wrote:
> > >>>>
> > >>>>> Hi Yu,
> > >>>>>
> > >>>>> Thanks for your updates. I am still working on the fix for
> > >> FLINK-16626
> > >>>> and
> > >>>>> it is expected to be completed by this Sunday after thorough
> testing.
> > >>>>>
> > >>>>> Sincerely,
> > >>>>> Weike
> > >>>>>
> > >>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> > >>>>>
> > >>>>>> Updates for 1.10.1 watched issues (we are in good progress and
> > >> almost
> > >>>>>> there
> > >>>>>> to produce the first RC, thanks all for the efforts):
> > >>>>>>
> > >>>>>> * Blockers (3 left)
> > >>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> > >> batch
> > >>>> job
> > >>>>>> (instead of AskTimeoutException)
> > >>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>>>>> repeated
> > >>>>>> job submission
> > >>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> > >> ClassNotFoundException
> > >>>> when
> > >>>>>> use flink-sql-connector-elasticsearch7
> > >>>>>> - [Closed] FLINK-16262 Class loader problem with
> > >>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> to
> > >>>>>> minimise its OutOfMemoryError
> > >>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> > >>> memory
> > >>>>>> state backends
> > >>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
> > >>>> cancelling a
> > >>>>>> job in yarn per-job mode
> > >>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster
> before
> > >>>>>> client
> > >>>>>> can retrieve JobResult
> > >>>>>> - [PR reviewed] FLINK-16913
> > >>>> ReadableConfigToConfigurationAdapter#getEnum
> > >>>>>> throws UnsupportedOperationException
> > >>>>>>
> > >>>>>> * Critical (1 left)
> > >>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> > >>> error
> > >>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>> results
> > >>>>>> with state clean up
> > >>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
> > >> unique
> > >>>> key
> > >>>>>> for UpsertStreamTableSink
> > >>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> > >>> should
> > >>>> be
> > >>>>>> handled as Fatal Error in TaskManager
> > >>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> > >> lifetime
> > >>>> of
> > >>>>>> a
> > >>>>>> slot
> > >>>>>>
> > >>>>>> Please let me know if you find any missing ones, thanks.
> > >>>>>>
> > >>>>>> Best Regards,
> > >>>>>> Yu
> > >>>>>>
> > >>>>>>
> > >>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
> > >>>>>>>
> > >>>>>>> * Blockers (4 left)
> > >>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> > >>>>>> submitting
> > >>>>>>> batch job (instead of AskTimeoutException)
> > >>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>>>>>> repeated job submission
> > >>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> > >>> ClassNotFoundException
> > >>>>>> when
> > >>>>>>> use flink-sql-connector-elasticsearch7
> > >>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
> > >>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> > >> to
> > >>>>>>> minimise its OutOfMemoryError
> > >>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
> > >>>> memory
> > >>>>>>> state backends
> > >>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> > >>>>>> MiniCluster
> > >>>>>>> before client can retrieve JobResult
> > >>>>>>>
> > >>>>>>> * Critical (2 left)
> > >>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>>> results
> > >>>>>>> with state clean up
> > >>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> > >> correct
> > >>>>>>> unique key for UpsertStreamTableSink
> > >>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
> > >> should
> > >>>> be
> > >>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
> > >> a
> > >>>> slot
> > >>>>>>>
> > >>>>>>> There's a new blocker added. Please let me know if you find any
> > >>> other
> > >>>>>>> issues should be put into the watch list. Thanks.
> > >>>>>>>
> > >>>>>>> Best Regards,
> > >>>>>>> Yu
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi All,
> > >>>>>>>>
> > >>>>>>>> Here is the status update of issues in 1.10.1 watch list:
> > >>>>>>>>
> > >>>>>>>> * Blockers (7)
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> > >>>>>>>> submitting batch job (instead of AskTimeoutException)
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> > >> OOM
> > >>>>>> error
> > >>>>>>>> on repeated job submission
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >>>>>>>> ClassNotFoundException when use
> > >> flink-sql-connector-elasticsearch7
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> > >>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> > >>> to
> > >>>>>>>> minimise its OutOfMemoryError
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
> > >> files
> > >>>>>>>>
> > >>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with memory
> > >>>> state
> > >>>>>>>> backends
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> * Critical (4)
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>>> results
> > >>>>>>>> with state clean up
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> > >>> correct
> > >>>>>>>> unique key for UpsertStreamTableSink
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> > >>> be
> > >>>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>>>
> > >>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
> > >> of a
> > >>>>>> slot
> > >>>>>>>>
> > >>>>>>>> Please let me know if you see any new blockers to add to the
> > >> list.
> > >>>>>> Thanks.
> > >>>>>>>>
> > >>>>>>>> Best Regards,
> > >>>>>>>> Yu
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Thanks for the updates Till!
> > >>>>>>>>>
> > >>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
> > >> and
> > >>>>>>>>> complete fix separately, and only include the easy one in
> > >> 1.10.1?
> > >>> Or
> > >>>>>> please
> > >>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
> > >>>>>> nothing"
> > >>>>>>>>> policy is preferred (smile). Thanks.
> > >>>>>>>>>
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Yu
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
> > >> trohrmann@apache.org
> > >>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
> > >> our
> > >>>>>> release
> > >>>>>>>>>> manager Yu.
> > >>>>>>>>>>
> > >>>>>>>>>> I think we can soon merge the increase of metaspace size and
> > >>>>>> improving
> > >>>>>>>>>> the
> > >>>>>>>>>> error message. The assumption is that we currently don't have
> > >> too
> > >>>>>> many
> > >>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> > >>> course,
> > >>>>>> the
> > >>>>>>>>>> sooner we release the bug fix release, the fewer deployments
> > >> will
> > >>>> be
> > >>>>>>>>>> affected by this change.
> > >>>>>>>>>>
> > >>>>>>>>>> For FLINK-16018, I think there would be an easy solution which
> > >> we
> > >>>>>> could
> > >>>>>>>>>> include in the bug fix release. The proper fix will most
> likely
> > >>>> take
> > >>>>>> a
> > >>>>>>>>>> bit
> > >>>>>>>>>> longer, though.
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Till
> > >>>>>>>>>>
> > >>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> > >>>>>> azagrebin@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> > >> user
> > >>>>>>>>>> mailing
> > >>>>>>>>>>> list
> > >>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> > >>>> Specifically
> > >>>>>>>>>> asking
> > >>>>>>>>>>>> for who has very small TM setups?
> > >>>>>>>>>>>
> > >>>>>>>>>>> There has been a survey about this topic since 10 days:
> > >>>>>>>>>>>
> > >>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
> > >> 1.10`
> > >>>>>>>>>>> I can ask there about the small TM setups.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> > >>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
> > >>> on
> > >>>>>>>>>> restore
> > >>>>>>>>>>> with
> > >>>>>>>>>>>> memory state backends
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
> > >>>>>> There're
> > >>>>>>>>>>> totally
> > >>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
> > >>>>>> which 14
> > >>>>>>>>>>>> already resolved and 31 left, and the below ones are
> > >> watched
> > >>>>>>>>>> (meaning
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> once the below ones got fixed, we will kick of the
> > >> releasing
> > >>>> with
> > >>>>>>>>>> left
> > >>>>>>>>>>> ones
> > >>>>>>>>>>>> postponed to 1.10.2 unless objections):
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> * Blockers (7)
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
> > >>> when
> > >>>>>>>>>>> submitting
> > >>>>>>>>>>>> batch job (instead of AskTimeoutException)
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
> > >>> Metaspace
> > >>>>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>>> on repeated job submission
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >>>>>>>>>>> ClassNotFoundException
> > >>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> > >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > >> directory
> > >>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
> > >>>>>> Metaspace to
> > >>>>>>>>>>>> minimise its OutOfMemoryError
> > >>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
> > >>>> files
> > >>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
> > >>>> memory
> > >>>>>>>>>> state
> > >>>>>>>>>>>> backends
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> * Critical (4)
> > >>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
> > >> aggregate
> > >>>>>>>>>> results
> > >>>>>>>>>>> with
> > >>>>>>>>>>>> state clean up
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
> > >> extract
> > >>>>>> correct
> > >>>>>>>>>>> unique
> > >>>>>>>>>>>> key for UpsertStreamTableSink
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> > >>>>>> should be
> > >>>>>>>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
> > >>> lifetime
> > >>>>>> of a
> > >>>>>>>>>> slot
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please raise your hand if you find any other issues should
> > >> be
> > >>>> put
> > >>>>>>>>>> into
> > >>>>>>>>>>> this
> > >>>>>>>>>>>> list, thanks.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please also note that 1.10.2 version is already created
> > >>> (thanks
> > >>>>>> for
> > >>>>>>>>>> the
> > >>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
> > >>> to
> > >>>> it
> > >>>>>>>>>> when
> > >>>>>>>>>>>> necessary.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>> Yu
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
> > >> sewen@apache.org
> > >>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> > >>> user
> > >>>>>>>>>> mailing
> > >>>>>>>>>>>> list
> > >>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> > >>>>>> Specifically
> > >>>>>>>>>>> asking
> > >>>>>>>>>>>>> for who has very small TM setups?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for the reminder Stephan and the
> > >> inputs/discussion
> > >>>>>>>>>> Andrey and
> > >>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
> > >>>>>>>>>> introduce new
> > >>>>>>>>>>>>>> issues when resolving the existing one. Will watch
> > >>> closely
> > >>>>>>>>>> about the
> > >>>>>>>>>>>>>> discussion and conclusion around metaspace
> > >> configuration.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
> > >>> optimistic
> > >>>>>> that
> > >>>>>>>>>>>>> reserving
> > >>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
> > >>> cause a
> > >>>>>> big
> > >>>>>>>>>>>> problem,
> > >>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
> > >>>>>> memory
> > >>>>>>>>>> per TM
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>>> the common case.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> > >>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
> > >>>>>> default
> > >>>>>>>>>> value
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
> > >>>> default
> > >>>>>>>>>>>>>>> 'flink-conf.yaml'.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> That means:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
> > >>> it's
> > >>>>>> not
> > >>>>>>>>>>> really
> > >>>>>>>>>>>>>> taking
> > >>>>>>>>>>>>>>> memory from heap / managed for metaspace.
> > >>>>>>>>>>>>>>> - For people using custom configuration files:
> > >>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
> > >>>> metaspace
> > >>>>>>>>>> does
> > >>>>>>>>>>> not
> > >>>>>>>>>>>>>> really
> > >>>>>>>>>>>>>>> affect the flink memory internal components
> > >>> (heap,
> > >>>>>>>>>> managed,
> > >>>>>>>>>>>>> etc.).
> > >>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
> > >>>> likely
> > >>>>>>>>>> active
> > >>>>>>>>>>>>>> setups),
> > >>>>>>>>>>>>>>> then they will have less heap / managed memory,
> > >>> or
> > >>>>>>>>>> worse,
> > >>>>>>>>>>> their
> > >>>>>>>>>>>>>>> configuration may not work without adjustment.
> > >>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
> > >> 4
> > >>> GB
> > >>>>>> or
> > >>>>>>>>>> more),
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
> > >>> more
> > >>>>>>>>>> memory.
> > >>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
> > >>> GB)
> > >>>>>> the
> > >>>>>>>>>>>> metaspace
> > >>>>>>>>>>>>>>> increase is more likely to cause problem.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> So basically only people have small 'process.size' in
> > >>>>>> custom
> > >>>>>>>>>> config
> > >>>>>>>>>>>>> file
> > >>>>>>>>>>>>>>> are really affected. I'm not sure what is the
> > >>> proportion
> > >>>> of
> > >>>>>>>>>> such
> > >>>>>>>>>>> use
> > >>>>>>>>>>>>>> cases
> > >>>>>>>>>>>>>>> though. (From questions asked on the user ML,
> > >> probably
> > >>>> not
> > >>>>>>>>>> much).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
> > >> should
> > >>>> not
> > >>>>>> go
> > >>>>>>>>>> into
> > >>>>>>>>>>>>>> 1.10.1,
> > >>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> We have to trade off the fact that we may break
> > >> some
> > >>>>>>>>>> deployments
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
> > >>>>>>>>>>>>>>>> I simply lack the data points / insight at the
> > >> moment
> > >>>> to
> > >>>>>>>>>>> understand
> > >>>>>>>>>>>>> how
> > >>>>>>>>>>>>>>>> significant both cases are, meaning how many users
> > >>>> would
> > >>>>>> be
> > >>>>>>>>>>>> affected
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> how badly.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Independent of that, improving the error message is
> > >>>>>> always
> > >>>>>>>>>>> helpful.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > >>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> > >> because
> > >>>>>> users
> > >>>>>>>>>> expect
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> upgrade
> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> > >>> be
> > >>>>>>>>>> possible
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> change.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
> > >> only
> > >>>>>> try to
> > >>>>>>>>>>>> improve
> > >>>>>>>>>>>>>>> error
> > >>>>>>>>>>>>>>>>> message and docs.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
> > >>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> > >> because
> > >>>>>> users
> > >>>>>>>>>> expect
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> upgrade
> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> > >>> be
> > >>>>>>>>>> possible
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> change.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
> > >> Zagrebin
> > >>> <
> > >>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> > >> Metaspace
> > >>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>> repeated
> > >>>>>>>>>>>>>>>>>>> job”
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
> > >>>> covered
> > >>>>>>>>>> by:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
> > >>> be
> > >>>>>>>>>> handled as
> > >>>>>>>>>>>>> Fatal
> > >>>>>>>>>>>>>>>> Error
> > >>>>>>>>>>>>>>>>>>> in TaskManager
> > >>>>>>>>>>>>>>>>>>> no full consensus there but improving error
> > >>>>>> message
> > >>>>>>>>>> for
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> task
> > >>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
> > >>>>>>>>>> Metaspace to
> > >>>>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>>>> see further
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
> > >>> "SdkMBeanRegistrySupport"
> > >>>>>> from
> > >>>>>>>>>>>>> dynamically
> > >>>>>>>>>>>>>>>>> loaded
> > >>>>>>>>>>>>>>>>>>> AWS connectors
> > >>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
> > >>>> close
> > >>>>>> to
> > >>>>>>>>>> be
> > >>>>>>>>>>>>> resolved
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>>> Metaspace"
> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> > >>>> bugfix
> > >>>>>>>>>> release?
> > >>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>> changes
> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> > >>>>>> memory
> > >>>>>>>>>> on
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> setups
> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
> > >>>>>> resolved
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> reported
> > >>>>>>>>>>>>>>>>> problems
> > >>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
> > >>> merged
> > >>>>>> it
> > >>>>>>>>>> today
> > >>>>>>>>>>> as
> > >>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>> were
> > >>>>>>>>>>>>>>>>>>> no more concerns.
> > >>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
> > >> message
> > >>>>>> error
> > >>>>>>>>>> with
> > >>>>>>>>>>>>>>> reference
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> the metaspace option should help the most
> > >>>>>>>>>>>>>>>>>>> because user would not have to read all docs
> > >> to
> > >>>> fix
> > >>>>>> it
> > >>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>> Andrey
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> > >> Metaspace
> > >>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>> repeated
> > >>>>>>>>>>>>>>>>>>> job"
> > >>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
> > >> exact
> > >>>>>>>>>> solution,
> > >>>>>>>>>>> yet,
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
> > >>> are
> > >>>>>>>>>> hard to
> > >>>>>>>>>>>>>> predict,
> > >>>>>>>>>>>>>>>> so I
> > >>>>>>>>>>>>>>>>>>> am
> > >>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>>> Metaspace"
> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> > >>>> bugfix
> > >>>>>>>>>> release?
> > >>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>> changes
> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> > >>>>>> memory
> > >>>>>>>>>> on
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> setups
> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
> > >>> two
> > >>>>>>>>>> issues
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> > >>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
> > >> eye
> > >>>> on
> > >>>>>>>>>> these
> > >>>>>>>>>>> two.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> > >>>>>>>>>> imjark@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
> > >> should
> > >>>> also
> > >>>>>>>>>> try to
> > >>>>>>>>>>>> get
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> following issues:
> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
> > >>>> flink-conf
> > >>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>> SQL
> > >>>>>>>>>>>>>>>>>>> CLI
> > >>>>>>>>>>>>>>>>>>>>>> environment
> > >>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
> > >>>>>>>>>> watermark
> > >>>>>>>>>>>>> interval,
> > >>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
> > >>>>>> uncatched
> > >>>>>>>>>>>>> exception
> > >>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>> thrown
> > >>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
> > >> using
> > >>>> SQL
> > >>>>>> CLI
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>> Jark
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> > >>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
> > >> added
> > >>>>>>>>>> 1.10.1 into
> > >>>>>>>>>>>>>>>> FLINK-16225
> > >>>>>>>>>>>>>>>>>>>>> fix
> > >>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
> > >>>> Critical.
> > >>>>>>>>>> Will
> > >>>>>>>>>>> also
> > >>>>>>>>>>>>>> watch
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> > >>> Zagrebin
> > >>>> <
> > >>>>>>>>>>>>>>>> azagrebin@apache.org
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
> > >>>>>>>>>> discussion!
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Apart from
> > >>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>> Metaspace
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
> > >>>>>> following
> > >>>>>>>>>>>> issues:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> > >>>> should
> > >>>>>> be
> > >>>>>>>>>>> handled
> > >>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> Fatal
> > >>>>>>>>>>>>>>>>>>>>>> Error
> > >>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>> TaskManager
> > >>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
> > >>> even
> > >>>>>> in a
> > >>>>>>>>>>> better
> > >>>>>>>>>>>>> way
> > >>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>> OOM
> > >>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
> > >>>>>> immediately
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
> > >> loader
> > >>>> to
> > >>>>>>>>>> lifetime
> > >>>>>>>>>>>> of
> > >>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>> slot
> > >>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
> > >>> against
> > >>>>>> class
> > >>>>>>>>>>>> loading
> > >>>>>>>>>>>>>>> leaks
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
> > >>> when
> > >>>>>>>>>> submitting
> > >>>>>>>>>>>>> batch
> > >>>>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>>>>>>>> (instead
> > >>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
> > >>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
> > >>>> multiple
> > >>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>> Andrey
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> > >>> Li <
> > >>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
> > >>> the
> > >>>>>>>>>> 1.10.1
> > >>>>>>>>>>>>> release.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
> > >> are
> > >>>>>> looking
> > >>>>>>>>>>>> forward
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> them.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> > >>> Guo <
> > >>>>>>>>>>>>>>> karmagyz@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
> > >> cycle.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> > >>>> Song
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
> > >>> get
> > >>>>>> some
> > >>>>>>>>>>> inputs
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>> prioritizing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
> > >>> good
> > >>>> to
> > >>>>>>>>>> me.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
> > >> Li <
> > >>>>>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
> > >>> patch
> > >>>>>>>>>> release
> > >>>>>>>>>>> of
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>>>> major
> > >>>>>>>>>>>>>>>>>>>>>>>> version
> > >>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
> > >>>>>>>>>> depending on
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> speed
> > >>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>> blocker
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
> > >> 28th
> > >>>>>> [1],
> > >>>>>>>>>>> released
> > >>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>> Jul
> > >>>>>>>>>>>>>>>>>>>>>> 3rd
> > >>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
> > >> 23rd
> > >>>>>> [3],
> > >>>>>>>>>>> released
> > >>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>> Oct
> > >>>>>>>>>>>>>>>>>>>>>> 19th
> > >>>>>>>>>>>>>>>>>>>>>>>> [4]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
> > >>>>>> course,
> > >>>>>>>>>> but
> > >>>>>>>>>>>> could
> > >>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>> as a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
> > >> let
> > >>> me
> > >>>>>>>>>> know if
> > >>>>>>>>>>> any
> > >>>>>>>>>>>>>>>>>>>>> concerns
> > >>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> > >>>> Song
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
> > >>>>>> volunteering
> > >>>>>>>>>> to be
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>> manager.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
> > >>>> plan
> > >>>>>> for
> > >>>>>>>>>>>> 1.10.1?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> > >>>> Zhijiang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
> > >> .invalid>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
> > >> Yu!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>> ------------------------------------------------------------------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
> > >> Flink
> > >>>>>> 1.10.1
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
> > >> Yes
> > >>>> will
> > >>>>>>>>>> also
> > >>>>>>>>>>> keep
> > >>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>> eye
> > >>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> > >>>>>> Cheng <
> > >>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
> > >>>> discussion
> > >>>>>> and
> > >>>>>>>>>>>>> volunteer
> > >>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
> > >>> issues[1]
> > >>>>>>>>>> which are
> > >>>>>>>>>>>>>> marked
> > >>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> > >>>> year
> > >>>>>> in
> > >>>>>>>>>>> NOTICE
> > >>>>>>>>>>>>>> files
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> > >>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> > >>>>>>>>>> and
> > >>>>>>>>>>>> usrlib
> > >>>>>>>>>>>>>>>>>>>>>>> directory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
> > >> SearchTemplateRequest
> > >>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
> > >>>>>>>>>>>>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
> > >>> reporting
> > >>>>>> when
> > >>>>>>>>>>>>> submitting
> > >>>>>>>>>>>>>>>>>>>>>> batch
> > >>>>>>>>>>>>>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
> > >>> in
> > >>>>>>>>>> 1.10.1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> > >>> Li
> > >>>> <
> > >>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
> > >>> helpful.
> > >>>>>>>>>> Thanks a
> > >>>>>>>>>>>> lot!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> > >>>> jincheng
> > >>>>>>>>>> sun <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
> > >> discussion
> > >>>>>> Yu. I
> > >>>>>>>>>> would
> > >>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>> give
> > >>>>>>>>>>>>>>>>>>>>>>>>>> you a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
> > >>>>>>>>>> finished.(If you
> > >>>>>>>>>>>>>>>>>>>>> need)
> > >>>>>>>>>>>>>>>>>>>>>>> :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Congxian Qiu <qc...@gmail.com>.
Thanks a lot for driving this, Yu! Looking forward to the first RC of 1.10.1
FLINK-16576 has been merged into release-1.10 already.
Best,
Congxian
Dian Fu <di...@gmail.com> 于2020年4月16日周四 上午10:35写道：
> Thanks a lot for driving this, Yu! Looking forward for the first RC of
> 1.10.1.
>
> > 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
> >
> > Looking forward the first RC of Flink 1.10.1 .
> > Good job Yu!
> >
> > Best,
> > Jincheng
> >
> >
> >
> > Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
> >
> >> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> >> released.
> >>
> >> Best,
> >> Jark
> >>
> >> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org>
> wrote:
> >>
> >>> Great to see that will have the first RC for Flink 1.10.1 soon. Thanks
> a
> >>> lot for driving this effort Yu!
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> >>>
> >>>> Thanks Weike and all others for the efforts!
> >>>>
> >>>> Here comes the latest status, we are in good shape and plan to produce
> >>> RC1
> >>>> next week.
> >>>>
> >>>> * Blockers (1 left)
> >>>> - [Closed] FLINK-16018 Improve error reporting when submitting batch
> >>> job
> >>>> (instead of AskTimeoutException)
> >>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>> repeated
> >>>> job submission
> >>>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> >>> when
> >>>> use flink-sql-connector-elasticsearch7
> >>>> - [Closed] FLINK-16262 Class loader problem with
> >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >>>> minimise its OutOfMemoryError
> >>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> >>> client
> >>>> can retrieve JobResult
> >>>> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> >>>> throws UnsupportedOperationException
> >>>> - [Closed] FLINK-16626 Exception encountered when cancelling a job in
> >>>> yarn per-job mode
> >>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
> >> the
> >>>> input column is of composite type
> >>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> >> memory
> >>>> state backends
> >>>>
> >>>> * Critical (1 left)
> >>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> >>>> with state clean up
> >>>> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> >> key
> >>>> for UpsertStreamTableSink
> >>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
> >>> be
> >>>> handled as Fatal Error in TaskManager
> >>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> >> error
> >>>> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> >>> of a
> >>>> slot
> >>>>
> >>>> Please let me know if any concerns/comments. Thanks.
> >>>>
> >>>> Best Regards,
> >>>> Yu
> >>>>
> >>>>
> >>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> >>> wrote:
> >>>>
> >>>>> Hi Yu,
> >>>>>
> >>>>> Thanks for your updates. I am still working on the fix for
> >> FLINK-16626
> >>>> and
> >>>>> it is expected to be completed by this Sunday after thorough testing.
> >>>>>
> >>>>> Sincerely,
> >>>>> Weike
> >>>>>
> >>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> >>>>>
> >>>>>> Updates for 1.10.1 watched issues (we are in good progress and
> >> almost
> >>>>>> there
> >>>>>> to produce the first RC, thanks all for the efforts):
> >>>>>>
> >>>>>> * Blockers (3 left)
> >>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> >> batch
> >>>> job
> >>>>>> (instead of AskTimeoutException)
> >>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>>>>> repeated
> >>>>>> job submission
> >>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> >> ClassNotFoundException
> >>>> when
> >>>>>> use flink-sql-connector-elasticsearch7
> >>>>>> - [Closed] FLINK-16262 Class loader problem with
> >>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >>>>>> minimise its OutOfMemoryError
> >>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> >>> memory
> >>>>>> state backends
> >>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
> >>>> cancelling a
> >>>>>> job in yarn per-job mode
> >>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> >>>>>> client
> >>>>>> can retrieve JobResult
> >>>>>> - [PR reviewed] FLINK-16913
> >>>> ReadableConfigToConfigurationAdapter#getEnum
> >>>>>> throws UnsupportedOperationException
> >>>>>>
> >>>>>> * Critical (1 left)
> >>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> >>> error
> >>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>> results
> >>>>>> with state clean up
> >>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
> >> unique
> >>>> key
> >>>>>> for UpsertStreamTableSink
> >>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> >>> should
> >>>> be
> >>>>>> handled as Fatal Error in TaskManager
> >>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> >> lifetime
> >>>> of
> >>>>>> a
> >>>>>> slot
> >>>>>>
> >>>>>> Please let me know if you find any missing ones, thanks.
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Yu
> >>>>>>
> >>>>>>
> >>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
> >>>>>>>
> >>>>>>> * Blockers (4 left)
> >>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> >>>>>> submitting
> >>>>>>> batch job (instead of AskTimeoutException)
> >>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>>>>>> repeated job submission
> >>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> >>> ClassNotFoundException
> >>>>>> when
> >>>>>>> use flink-sql-connector-elasticsearch7
> >>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
> >>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> >> to
> >>>>>>> minimise its OutOfMemoryError
> >>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
> >>>> memory
> >>>>>>> state backends
> >>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> >>>>>> MiniCluster
> >>>>>>> before client can retrieve JobResult
> >>>>>>>
> >>>>>>> * Critical (2 left)
> >>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>>> results
> >>>>>>> with state clean up
> >>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> >> correct
> >>>>>>> unique key for UpsertStreamTableSink
> >>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
> >> should
> >>>> be
> >>>>>>> handled as Fatal Error in TaskManager
> >>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
> >> a
> >>>> slot
> >>>>>>>
> >>>>>>> There's a new blocker added. Please let me know if you find any
> >>> other
> >>>>>>> issues should be put into the watch list. Thanks.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Yu
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> Here is the status update of issues in 1.10.1 watch list:
> >>>>>>>>
> >>>>>>>> * Blockers (7)
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> >>>>>>>> submitting batch job (instead of AskTimeoutException)
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> >> OOM
> >>>>>> error
> >>>>>>>> on repeated job submission
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >>>>>>>> ClassNotFoundException when use
> >> flink-sql-connector-elasticsearch7
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> >>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> >>> to
> >>>>>>>> minimise its OutOfMemoryError
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
> >> files
> >>>>>>>>
> >>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with memory
> >>>> state
> >>>>>>>> backends
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> * Critical (4)
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>>> results
> >>>>>>>> with state clean up
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> >>> correct
> >>>>>>>> unique key for UpsertStreamTableSink
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> >>> be
> >>>>>>>> handled as Fatal Error in TaskManager
> >>>>>>>>
> >>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
> >> of a
> >>>>>> slot
> >>>>>>>>
> >>>>>>>> Please let me know if you see any new blockers to add to the
> >> list.
> >>>>>> Thanks.
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Yu
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks for the updates Till!
> >>>>>>>>>
> >>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
> >> and
> >>>>>>>>> complete fix separately, and only include the easy one in
> >> 1.10.1?
> >>> Or
> >>>>>> please
> >>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
> >>>>>> nothing"
> >>>>>>>>> policy is preferred (smile). Thanks.
> >>>>>>>>>
> >>>>>>>>> Best Regards,
> >>>>>>>>> Yu
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
> >> trohrmann@apache.org
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
> >> our
> >>>>>> release
> >>>>>>>>>> manager Yu.
> >>>>>>>>>>
> >>>>>>>>>> I think we can soon merge the increase of metaspace size and
> >>>>>> improving
> >>>>>>>>>> the
> >>>>>>>>>> error message. The assumption is that we currently don't have
> >> too
> >>>>>> many
> >>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> >>> course,
> >>>>>> the
> >>>>>>>>>> sooner we release the bug fix release, the fewer deployments
> >> will
> >>>> be
> >>>>>>>>>> affected by this change.
> >>>>>>>>>>
> >>>>>>>>>> For FLINK-16018, I think there would be an easy solution which
> >> we
> >>>>>> could
> >>>>>>>>>> include in the bug fix release. The proper fix will most likely
> >>>> take
> >>>>>> a
> >>>>>>>>>> bit
> >>>>>>>>>> longer, though.
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Till
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> >>>>>> azagrebin@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> >> user
> >>>>>>>>>> mailing
> >>>>>>>>>>> list
> >>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> >>>> Specifically
> >>>>>>>>>> asking
> >>>>>>>>>>>> for who has very small TM setups?
> >>>>>>>>>>>
> >>>>>>>>>>> There has been a survey about this topic since 10 days:
> >>>>>>>>>>>
> >>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
> >> 1.10`
> >>>>>>>>>>> I can ask there about the small TM setups.
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> >>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
> >>> on
> >>>>>>>>>> restore
> >>>>>>>>>>> with
> >>>>>>>>>>>> memory state backends
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
> >>>>>> There're
> >>>>>>>>>>> totally
> >>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
> >>>>>> which 14
> >>>>>>>>>>>> already resolved and 31 left, and the below ones are
> >> watched
> >>>>>>>>>> (meaning
> >>>>>>>>>>> that
> >>>>>>>>>>>> once the below ones got fixed, we will kick of the
> >> releasing
> >>>> with
> >>>>>>>>>> left
> >>>>>>>>>>> ones
> >>>>>>>>>>>> postponed to 1.10.2 unless objections):
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Blockers (7)
> >>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
> >>> when
> >>>>>>>>>>> submitting
> >>>>>>>>>>>> batch job (instead of AskTimeoutException)
> >>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
> >>> Metaspace
> >>>>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>>> on repeated job submission
> >>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >>>>>>>>>>> ClassNotFoundException
> >>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
> >>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> >> directory
> >>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
> >>>>>> Metaspace to
> >>>>>>>>>>>> minimise its OutOfMemoryError
> >>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
> >>>> files
> >>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
> >>>> memory
> >>>>>>>>>> state
> >>>>>>>>>>>> backends
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Critical (4)
> >>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
> >> aggregate
> >>>>>>>>>> results
> >>>>>>>>>>> with
> >>>>>>>>>>>> state clean up
> >>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
> >> extract
> >>>>>> correct
> >>>>>>>>>>> unique
> >>>>>>>>>>>> key for UpsertStreamTableSink
> >>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> >>>>>> should be
> >>>>>>>>>>>> handled as Fatal Error in TaskManager
> >>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
> >>> lifetime
> >>>>>> of a
> >>>>>>>>>> slot
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please raise your hand if you find any other issues should
> >> be
> >>>> put
> >>>>>>>>>> into
> >>>>>>>>>>> this
> >>>>>>>>>>>> list, thanks.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please also note that 1.10.2 version is already created
> >>> (thanks
> >>>>>> for
> >>>>>>>>>> the
> >>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
> >>> to
> >>>> it
> >>>>>>>>>> when
> >>>>>>>>>>>> necessary.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Yu
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
> >> sewen@apache.org
> >>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> >>> user
> >>>>>>>>>> mailing
> >>>>>>>>>>>> list
> >>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> >>>>>> Specifically
> >>>>>>>>>>> asking
> >>>>>>>>>>>>> for who has very small TM setups?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the reminder Stephan and the
> >> inputs/discussion
> >>>>>>>>>> Andrey and
> >>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
> >>>>>>>>>> introduce new
> >>>>>>>>>>>>>> issues when resolving the existing one. Will watch
> >>> closely
> >>>>>>>>>> about the
> >>>>>>>>>>>>>> discussion and conclusion around metaspace
> >> configuration.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
> >>> optimistic
> >>>>>> that
> >>>>>>>>>>>>> reserving
> >>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
> >>> cause a
> >>>>>> big
> >>>>>>>>>>>> problem,
> >>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
> >>>>>> memory
> >>>>>>>>>> per TM
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>> the common case.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> >>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
> >>>>>> default
> >>>>>>>>>> value
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
> >>>> default
> >>>>>>>>>>>>>>> 'flink-conf.yaml'.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> That means:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
> >>> it's
> >>>>>> not
> >>>>>>>>>>> really
> >>>>>>>>>>>>>> taking
> >>>>>>>>>>>>>>> memory from heap / managed for metaspace.
> >>>>>>>>>>>>>>> - For people using custom configuration files:
> >>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
> >>>> metaspace
> >>>>>>>>>> does
> >>>>>>>>>>> not
> >>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>> affect the flink memory internal components
> >>> (heap,
> >>>>>>>>>> managed,
> >>>>>>>>>>>>> etc.).
> >>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
> >>>> likely
> >>>>>>>>>> active
> >>>>>>>>>>>>>> setups),
> >>>>>>>>>>>>>>> then they will have less heap / managed memory,
> >>> or
> >>>>>>>>>> worse,
> >>>>>>>>>>> their
> >>>>>>>>>>>>>>> configuration may not work without adjustment.
> >>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
> >> 4
> >>> GB
> >>>>>> or
> >>>>>>>>>> more),
> >>>>>>>>>>>> it
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
> >>> more
> >>>>>>>>>> memory.
> >>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
> >>> GB)
> >>>>>> the
> >>>>>>>>>>>> metaspace
> >>>>>>>>>>>>>>> increase is more likely to cause problem.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So basically only people have small 'process.size' in
> >>>>>> custom
> >>>>>>>>>> config
> >>>>>>>>>>>>> file
> >>>>>>>>>>>>>>> are really affected. I'm not sure what is the
> >>> proportion
> >>>> of
> >>>>>>>>>> such
> >>>>>>>>>>> use
> >>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>> though. (From questions asked on the user ML,
> >> probably
> >>>> not
> >>>>>>>>>> much).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
> >> should
> >>>> not
> >>>>>> go
> >>>>>>>>>> into
> >>>>>>>>>>>>>> 1.10.1,
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We have to trade off the fact that we may break
> >> some
> >>>>>>>>>> deployments
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
> >>>>>>>>>>>>>>>> I simply lack the data points / insight at the
> >> moment
> >>>> to
> >>>>>>>>>>> understand
> >>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>> significant both cases are, meaning how many users
> >>>> would
> >>>>>> be
> >>>>>>>>>>>> affected
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> how badly.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Independent of that, improving the error message is
> >>>>>> always
> >>>>>>>>>>> helpful.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> >>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> >> because
> >>>>>> users
> >>>>>>>>>> expect
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> upgrade
> >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> >>> be
> >>>>>>>>>> possible
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> change.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
> >> only
> >>>>>> try to
> >>>>>>>>>>>> improve
> >>>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>> message and docs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
> >>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
> >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> >> because
> >>>>>> users
> >>>>>>>>>> expect
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> upgrade
> >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> >>> be
> >>>>>>>>>> possible
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> change.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
> >> Zagrebin
> >>> <
> >>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> >> Metaspace
> >>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>> repeated
> >>>>>>>>>>>>>>>>>>> job”
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
> >>>> covered
> >>>>>>>>>> by:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
> >>> be
> >>>>>>>>>> handled as
> >>>>>>>>>>>>> Fatal
> >>>>>>>>>>>>>>>> Error
> >>>>>>>>>>>>>>>>>>> in TaskManager
> >>>>>>>>>>>>>>>>>>> no full consensus there but improving error
> >>>>>> message
> >>>>>>>>>> for
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> task
> >>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
> >>>>>>>>>> Metaspace to
> >>>>>>>>>>>>>> minimise
> >>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>>>> see further
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
> >>> "SdkMBeanRegistrySupport"
> >>>>>> from
> >>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>> loaded
> >>>>>>>>>>>>>>>>>>> AWS connectors
> >>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
> >>>> close
> >>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>>> Metaspace"
> >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> >>>> bugfix
> >>>>>>>>>> release?
> >>>>>>>>>>>> It
> >>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> >>>>>> memory
> >>>>>>>>>> on
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> setups
> >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
> >>>>>> resolved
> >>>>>>>>>> the
> >>>>>>>>>>>>> reported
> >>>>>>>>>>>>>>>>> problems
> >>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
> >>> merged
> >>>>>> it
> >>>>>>>>>> today
> >>>>>>>>>>> as
> >>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> were
> >>>>>>>>>>>>>>>>>>> no more concerns.
> >>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
> >> message
> >>>>>> error
> >>>>>>>>>> with
> >>>>>>>>>>>>>>> reference
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the metaspace option should help the most
> >>>>>>>>>>>>>>>>>>> because user would not have to read all docs
> >> to
> >>>> fix
> >>>>>> it
> >>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>> Andrey
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> >> Metaspace
> >>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>> repeated
> >>>>>>>>>>>>>>>>>>> job"
> >>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
> >> exact
> >>>>>>>>>> solution,
> >>>>>>>>>>> yet,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
> >>> are
> >>>>>>>>>> hard to
> >>>>>>>>>>>>>> predict,
> >>>>>>>>>>>>>>>> so I
> >>>>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>>> Metaspace"
> >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> >>>> bugfix
> >>>>>>>>>> release?
> >>>>>>>>>>>> It
> >>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> >>>>>> memory
> >>>>>>>>>> on
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> setups
> >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
> >>> two
> >>>>>>>>>> issues
> >>>>>>>>>>> and
> >>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> >>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
> >> eye
> >>>> on
> >>>>>>>>>> these
> >>>>>>>>>>> two.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> >>>>>>>>>> imjark@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
> >> should
> >>>> also
> >>>>>>>>>> try to
> >>>>>>>>>>>> get
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> following issues:
> >>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
> >>>> flink-conf
> >>>>>>>>>>>> parameters
> >>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>> SQL
> >>>>>>>>>>>>>>>>>>> CLI
> >>>>>>>>>>>>>>>>>>>>>> environment
> >>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
> >>>>>>>>>> watermark
> >>>>>>>>>>>>> interval,
> >>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
> >>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
> >>>>>> uncatched
> >>>>>>>>>>>>> exception
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> thrown
> >>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
> >> using
> >>>> SQL
> >>>>>> CLI
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>> Jark
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> >>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
> >> added
> >>>>>>>>>> 1.10.1 into
> >>>>>>>>>>>>>>>> FLINK-16225
> >>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
> >>>> Critical.
> >>>>>>>>>> Will
> >>>>>>>>>>> also
> >>>>>>>>>>>>>> watch
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> >>> Zagrebin
> >>>> <
> >>>>>>>>>>>>>>>> azagrebin@apache.org
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
> >>>>>>>>>> discussion!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Apart from
> >>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>> Metaspace
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> minimise
> >>>>>>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
> >>>>>> following
> >>>>>>>>>>>> issues:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> >>>> should
> >>>>>> be
> >>>>>>>>>>> handled
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> Fatal
> >>>>>>>>>>>>>>>>>>>>>> Error
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> TaskManager
> >>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
> >>> even
> >>>>>> in a
> >>>>>>>>>>> better
> >>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>> OOM
> >>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
> >>>>>> immediately
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
> >> loader
> >>>> to
> >>>>>>>>>> lifetime
> >>>>>>>>>>>> of
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> slot
> >>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
> >>> against
> >>>>>> class
> >>>>>>>>>>>> loading
> >>>>>>>>>>>>>>> leaks
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
> >>> when
> >>>>>>>>>> submitting
> >>>>>>>>>>>>> batch
> >>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>> (instead
> >>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
> >>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
> >>>> multiple
> >>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>> Andrey
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> >>> Li <
> >>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
> >>> the
> >>>>>>>>>> 1.10.1
> >>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
> >> are
> >>>>>> looking
> >>>>>>>>>>>> forward
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> >>> Guo <
> >>>>>>>>>>>>>>> karmagyz@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
> >> cycle.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> >>>> Song
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
> >>> get
> >>>>>> some
> >>>>>>>>>>> inputs
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> prioritizing
> >>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
> >>> good
> >>>> to
> >>>>>>>>>> me.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
> >> Li <
> >>>>>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
> >>> patch
> >>>>>>>>>> release
> >>>>>>>>>>> of
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>> major
> >>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
> >>>>>>>>>> depending on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> blocker
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
> >> 28th
> >>>>>> [1],
> >>>>>>>>>>> released
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> Jul
> >>>>>>>>>>>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
> >> 23rd
> >>>>>> [3],
> >>>>>>>>>>> released
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> Oct
> >>>>>>>>>>>>>>>>>>>>>> 19th
> >>>>>>>>>>>>>>>>>>>>>>>> [4]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
> >>>>>> course,
> >>>>>>>>>> but
> >>>>>>>>>>>> could
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
> >> let
> >>> me
> >>>>>>>>>> know if
> >>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>> concerns
> >>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> >>>> Song
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
> >>>>>> volunteering
> >>>>>>>>>> to be
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>> manager.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
> >>>> plan
> >>>>>> for
> >>>>>>>>>>>> 1.10.1?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> >>>> Zhijiang
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
> >> .invalid>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
> >> Yu!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> ------------------------------------------------------------------
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
> >> Flink
> >>>>>> 1.10.1
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
> >> Yes
> >>>> will
> >>>>>>>>>> also
> >>>>>>>>>>> keep
> >>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>> eye
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> >>>>>> Cheng <
> >>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
> >>>> discussion
> >>>>>> and
> >>>>>>>>>>>>> volunteer
> >>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
> >>> issues[1]
> >>>>>>>>>> which are
> >>>>>>>>>>>>>> marked
> >>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> >>>> year
> >>>>>> in
> >>>>>>>>>>> NOTICE
> >>>>>>>>>>>>>> files
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> >>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> >>>>>>>>>> and
> >>>>>>>>>>>> usrlib
> >>>>>>>>>>>>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
> >> SearchTemplateRequest
> >>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
> >>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
> >>> reporting
> >>>>>> when
> >>>>>>>>>>>>> submitting
> >>>>>>>>>>>>>>>>>>>>>> batch
> >>>>>>>>>>>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
> >>> in
> >>>>>>>>>> 1.10.1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> >>> Li
> >>>> <
> >>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
> >>> helpful.
> >>>>>>>>>> Thanks a
> >>>>>>>>>>>> lot!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> >>>> jincheng
> >>>>>>>>>> sun <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
> >> discussion
> >>>>>> Yu. I
> >>>>>>>>>> would
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> give
> >>>>>>>>>>>>>>>>>>>>>>>>>> you a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
> >>>>>>>>>> finished.(If you
> >>>>>>>>>>>>>>>>>>>>> need)
> >>>>>>>>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by godfrey he <go...@gmail.com>.
Thanks a lot for driving this, Yu!
Some users are already asking "where to download release-1.10.1?"
Looking forward to the rc.
Best,
Godfrey
Hequn Cheng <he...@apache.org> 于2020年4月16日周四 下午12:45写道：
> Thanks a lot for your great work Yu! Looking forward to the RC.
>
> Best, Hequn
>
> On Thu, Apr 16, 2020 at 10:35 AM Dian Fu <di...@gmail.com> wrote:
>
> > Thanks a lot for driving this, Yu! Looking forward for the first RC of
> > 1.10.1.
> >
> > > 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
> > >
> > > Looking forward the first RC of Flink 1.10.1 .
> > > Good job Yu!
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > >
> > > Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
> > >
> > >> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> > >> released.
> > >>
> > >> Best,
> > >> Jark
> > >>
> > >> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org>
> > wrote:
> > >>
> > >>> Great to see that will have the first RC for Flink 1.10.1 soon.
> Thanks
> > a
> > >>> lot for driving this effort Yu!
> > >>>
> > >>> Cheers,
> > >>> Till
> > >>>
> > >>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> > >>>
> > >>>> Thanks Weike and all others for the efforts!
> > >>>>
> > >>>> Here comes the latest status, we are in good shape and plan to
> produce
> > >>> RC1
> > >>>> next week.
> > >>>>
> > >>>> * Blockers (1 left)
> > >>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> batch
> > >>> job
> > >>>> (instead of AskTimeoutException)
> > >>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>> repeated
> > >>>> job submission
> > >>>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> > >>> when
> > >>>> use flink-sql-connector-elasticsearch7
> > >>>> - [Closed] FLINK-16262 Class loader problem with
> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > >>>> minimise its OutOfMemoryError
> > >>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> > >>> client
> > >>>> can retrieve JobResult
> > >>>> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> > >>>> throws UnsupportedOperationException
> > >>>> - [Closed] FLINK-16626 Exception encountered when cancelling a job
> in
> > >>>> yarn per-job mode
> > >>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
> > >> the
> > >>>> input column is of composite type
> > >>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> > >> memory
> > >>>> state backends
> > >>>>
> > >>>> * Critical (1 left)
> > >>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> results
> > >>>> with state clean up
> > >>>> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> > >> key
> > >>>> for UpsertStreamTableSink
> > >>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> should
> > >>> be
> > >>>> handled as Fatal Error in TaskManager
> > >>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> > >> error
> > >>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> lifetime
> > >>> of a
> > >>>> slot
> > >>>>
> > >>>> Please let me know if any concerns/comments. Thanks.
> > >>>>
> > >>>> Best Regards,
> > >>>> Yu
> > >>>>
> > >>>>
> > >>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> > >>> wrote:
> > >>>>
> > >>>>> Hi Yu,
> > >>>>>
> > >>>>> Thanks for your updates. I am still working on the fix for
> > >> FLINK-16626
> > >>>> and
> > >>>>> it is expected to be completed by this Sunday after thorough
> testing.
> > >>>>>
> > >>>>> Sincerely,
> > >>>>> Weike
> > >>>>>
> > >>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> > >>>>>
> > >>>>>> Updates for 1.10.1 watched issues (we are in good progress and
> > >> almost
> > >>>>>> there
> > >>>>>> to produce the first RC, thanks all for the efforts):
> > >>>>>>
> > >>>>>> * Blockers (3 left)
> > >>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> > >> batch
> > >>>> job
> > >>>>>> (instead of AskTimeoutException)
> > >>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>>>>> repeated
> > >>>>>> job submission
> > >>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> > >> ClassNotFoundException
> > >>>> when
> > >>>>>> use flink-sql-connector-elasticsearch7
> > >>>>>> - [Closed] FLINK-16262 Class loader problem with
> > >>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> to
> > >>>>>> minimise its OutOfMemoryError
> > >>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> > >>> memory
> > >>>>>> state backends
> > >>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
> > >>>> cancelling a
> > >>>>>> job in yarn per-job mode
> > >>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster
> before
> > >>>>>> client
> > >>>>>> can retrieve JobResult
> > >>>>>> - [PR reviewed] FLINK-16913
> > >>>> ReadableConfigToConfigurationAdapter#getEnum
> > >>>>>> throws UnsupportedOperationException
> > >>>>>>
> > >>>>>> * Critical (1 left)
> > >>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> > >>> error
> > >>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>> results
> > >>>>>> with state clean up
> > >>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
> > >> unique
> > >>>> key
> > >>>>>> for UpsertStreamTableSink
> > >>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> > >>> should
> > >>>> be
> > >>>>>> handled as Fatal Error in TaskManager
> > >>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> > >> lifetime
> > >>>> of
> > >>>>>> a
> > >>>>>> slot
> > >>>>>>
> > >>>>>> Please let me know if you find any missing ones, thanks.
> > >>>>>>
> > >>>>>> Best Regards,
> > >>>>>> Yu
> > >>>>>>
> > >>>>>>
> > >>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
> > >>>>>>>
> > >>>>>>> * Blockers (4 left)
> > >>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> > >>>>>> submitting
> > >>>>>>> batch job (instead of AskTimeoutException)
> > >>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >>>>>>> repeated job submission
> > >>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> > >>> ClassNotFoundException
> > >>>>>> when
> > >>>>>>> use flink-sql-connector-elasticsearch7
> > >>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
> > >>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> > >> to
> > >>>>>>> minimise its OutOfMemoryError
> > >>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
> > >>>> memory
> > >>>>>>> state backends
> > >>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> > >>>>>> MiniCluster
> > >>>>>>> before client can retrieve JobResult
> > >>>>>>>
> > >>>>>>> * Critical (2 left)
> > >>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>>> results
> > >>>>>>> with state clean up
> > >>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> > >> correct
> > >>>>>>> unique key for UpsertStreamTableSink
> > >>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
> > >> should
> > >>>> be
> > >>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
> > >> a
> > >>>> slot
> > >>>>>>>
> > >>>>>>> There's a new blocker added. Please let me know if you find any
> > >>> other
> > >>>>>>> issues should be put into the watch list. Thanks.
> > >>>>>>>
> > >>>>>>> Best Regards,
> > >>>>>>> Yu
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi All,
> > >>>>>>>>
> > >>>>>>>> Here is the status update of issues in 1.10.1 watch list:
> > >>>>>>>>
> > >>>>>>>> * Blockers (7)
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> > >>>>>>>> submitting batch job (instead of AskTimeoutException)
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> > >> OOM
> > >>>>>> error
> > >>>>>>>> on repeated job submission
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >>>>>>>> ClassNotFoundException when use
> > >> flink-sql-connector-elasticsearch7
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> > >>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> > >>> to
> > >>>>>>>> minimise its OutOfMemoryError
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
> > >> files
> > >>>>>>>>
> > >>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with memory
> > >>>> state
> > >>>>>>>> backends
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> * Critical (4)
> > >>>>>>>>
> > >>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > >>>> results
> > >>>>>>>> with state clean up
> > >>>>>>>>
> > >>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> > >>> correct
> > >>>>>>>> unique key for UpsertStreamTableSink
> > >>>>>>>>
> > >>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> > >>> be
> > >>>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>>>
> > >>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
> > >> of a
> > >>>>>> slot
> > >>>>>>>>
> > >>>>>>>> Please let me know if you see any new blockers to add to the
> > >> list.
> > >>>>>> Thanks.
> > >>>>>>>>
> > >>>>>>>> Best Regards,
> > >>>>>>>> Yu
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Thanks for the updates Till!
> > >>>>>>>>>
> > >>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
> > >> and
> > >>>>>>>>> complete fix separately, and only include the easy one in
> > >> 1.10.1?
> > >>> Or
> > >>>>>> please
> > >>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
> > >>>>>> nothing"
> > >>>>>>>>> policy is preferred (smile). Thanks.
> > >>>>>>>>>
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Yu
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
> > >> trohrmann@apache.org
> > >>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
> > >> our
> > >>>>>> release
> > >>>>>>>>>> manager Yu.
> > >>>>>>>>>>
> > >>>>>>>>>> I think we can soon merge the increase of metaspace size and
> > >>>>>> improving
> > >>>>>>>>>> the
> > >>>>>>>>>> error message. The assumption is that we currently don't have
> > >> too
> > >>>>>> many
> > >>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> > >>> course,
> > >>>>>> the
> > >>>>>>>>>> sooner we release the bug fix release, the fewer deployments
> > >> will
> > >>>> be
> > >>>>>>>>>> affected by this change.
> > >>>>>>>>>>
> > >>>>>>>>>> For FLINK-16018, I think there would be an easy solution which
> > >> we
> > >>>>>> could
> > >>>>>>>>>> include in the bug fix release. The proper fix will most
> likely
> > >>>> take
> > >>>>>> a
> > >>>>>>>>>> bit
> > >>>>>>>>>> longer, though.
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Till
> > >>>>>>>>>>
> > >>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> > >>>>>> azagrebin@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> > >> user
> > >>>>>>>>>> mailing
> > >>>>>>>>>>> list
> > >>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> > >>>> Specifically
> > >>>>>>>>>> asking
> > >>>>>>>>>>>> for who has very small TM setups?
> > >>>>>>>>>>>
> > >>>>>>>>>>> There has been a survey about this topic since 10 days:
> > >>>>>>>>>>>
> > >>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
> > >> 1.10`
> > >>>>>>>>>>> I can ask there about the small TM setups.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> > >>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
> > >>> on
> > >>>>>>>>>> restore
> > >>>>>>>>>>> with
> > >>>>>>>>>>>> memory state backends
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
> > >>>>>> There're
> > >>>>>>>>>>> totally
> > >>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
> > >>>>>> which 14
> > >>>>>>>>>>>> already resolved and 31 left, and the below ones are
> > >> watched
> > >>>>>>>>>> (meaning
> > >>>>>>>>>>> that
> > >>>>>>>>>>>> once the below ones got fixed, we will kick of the
> > >> releasing
> > >>>> with
> > >>>>>>>>>> left
> > >>>>>>>>>>> ones
> > >>>>>>>>>>>> postponed to 1.10.2 unless objections):
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> * Blockers (7)
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
> > >>> when
> > >>>>>>>>>>> submitting
> > >>>>>>>>>>>> batch job (instead of AskTimeoutException)
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
> > >>> Metaspace
> > >>>>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>>> on repeated job submission
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >>>>>>>>>>> ClassNotFoundException
> > >>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> > >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > >> directory
> > >>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
> > >>>>>> Metaspace to
> > >>>>>>>>>>>> minimise its OutOfMemoryError
> > >>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
> > >>>> files
> > >>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
> > >>>> memory
> > >>>>>>>>>> state
> > >>>>>>>>>>>> backends
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> * Critical (4)
> > >>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
> > >> aggregate
> > >>>>>>>>>> results
> > >>>>>>>>>>> with
> > >>>>>>>>>>>> state clean up
> > >>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
> > >> extract
> > >>>>>> correct
> > >>>>>>>>>>> unique
> > >>>>>>>>>>>> key for UpsertStreamTableSink
> > >>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> > >>>>>> should be
> > >>>>>>>>>>>> handled as Fatal Error in TaskManager
> > >>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
> > >>> lifetime
> > >>>>>> of a
> > >>>>>>>>>> slot
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please raise your hand if you find any other issues should
> > >> be
> > >>>> put
> > >>>>>>>>>> into
> > >>>>>>>>>>> this
> > >>>>>>>>>>>> list, thanks.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please also note that 1.10.2 version is already created
> > >>> (thanks
> > >>>>>> for
> > >>>>>>>>>> the
> > >>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
> > >>> to
> > >>>> it
> > >>>>>>>>>> when
> > >>>>>>>>>>>> necessary.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>> Yu
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
> > >> sewen@apache.org
> > >>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> > >>> user
> > >>>>>>>>>> mailing
> > >>>>>>>>>>>> list
> > >>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> > >>>>>> Specifically
> > >>>>>>>>>>> asking
> > >>>>>>>>>>>>> for who has very small TM setups?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for the reminder Stephan and the
> > >> inputs/discussion
> > >>>>>>>>>> Andrey and
> > >>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
> > >>>>>>>>>> introduce new
> > >>>>>>>>>>>>>> issues when resolving the existing one. Will watch
> > >>> closely
> > >>>>>>>>>> about the
> > >>>>>>>>>>>>>> discussion and conclusion around metaspace
> > >> configuration.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
> > >>> optimistic
> > >>>>>> that
> > >>>>>>>>>>>>> reserving
> > >>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
> > >>> cause a
> > >>>>>> big
> > >>>>>>>>>>>> problem,
> > >>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
> > >>>>>> memory
> > >>>>>>>>>> per TM
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>>> the common case.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> > >>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
> > >>>>>> default
> > >>>>>>>>>> value
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
> > >>>> default
> > >>>>>>>>>>>>>>> 'flink-conf.yaml'.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> That means:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
> > >>> it's
> > >>>>>> not
> > >>>>>>>>>>> really
> > >>>>>>>>>>>>>> taking
> > >>>>>>>>>>>>>>> memory from heap / managed for metaspace.
> > >>>>>>>>>>>>>>> - For people using custom configuration files:
> > >>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
> > >>>> metaspace
> > >>>>>>>>>> does
> > >>>>>>>>>>> not
> > >>>>>>>>>>>>>> really
> > >>>>>>>>>>>>>>> affect the flink memory internal components
> > >>> (heap,
> > >>>>>>>>>> managed,
> > >>>>>>>>>>>>> etc.).
> > >>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
> > >>>> likely
> > >>>>>>>>>> active
> > >>>>>>>>>>>>>> setups),
> > >>>>>>>>>>>>>>> then they will have less heap / managed memory,
> > >>> or
> > >>>>>>>>>> worse,
> > >>>>>>>>>>> their
> > >>>>>>>>>>>>>>> configuration may not work without adjustment.
> > >>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
> > >> 4
> > >>> GB
> > >>>>>> or
> > >>>>>>>>>> more),
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
> > >>> more
> > >>>>>>>>>> memory.
> > >>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
> > >>> GB)
> > >>>>>> the
> > >>>>>>>>>>>> metaspace
> > >>>>>>>>>>>>>>> increase is more likely to cause problem.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> So basically only people have small 'process.size' in
> > >>>>>> custom
> > >>>>>>>>>> config
> > >>>>>>>>>>>>> file
> > >>>>>>>>>>>>>>> are really affected. I'm not sure what is the
> > >>> proportion
> > >>>> of
> > >>>>>>>>>> such
> > >>>>>>>>>>> use
> > >>>>>>>>>>>>>> cases
> > >>>>>>>>>>>>>>> though. (From questions asked on the user ML,
> > >> probably
> > >>>> not
> > >>>>>>>>>> much).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
> > >> should
> > >>>> not
> > >>>>>> go
> > >>>>>>>>>> into
> > >>>>>>>>>>>>>> 1.10.1,
> > >>>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> We have to trade off the fact that we may break
> > >> some
> > >>>>>>>>>> deployments
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
> > >>>>>>>>>>>>>>>> I simply lack the data points / insight at the
> > >> moment
> > >>>> to
> > >>>>>>>>>>> understand
> > >>>>>>>>>>>>> how
> > >>>>>>>>>>>>>>>> significant both cases are, meaning how many users
> > >>>> would
> > >>>>>> be
> > >>>>>>>>>>>> affected
> > >>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> how badly.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Independent of that, improving the error message is
> > >>>>>> always
> > >>>>>>>>>>> helpful.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > >>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> > >> because
> > >>>>>> users
> > >>>>>>>>>> expect
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> upgrade
> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> > >>> be
> > >>>>>>>>>> possible
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> change.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
> > >> only
> > >>>>>> try to
> > >>>>>>>>>>>> improve
> > >>>>>>>>>>>>>>> error
> > >>>>>>>>>>>>>>>>> message and docs.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
> > >>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
> > >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> > >> because
> > >>>>>> users
> > >>>>>>>>>> expect
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> upgrade
> > >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> > >>> be
> > >>>>>>>>>> possible
> > >>>>>>>>>>>> with
> > >>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>> change.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
> > >> Zagrebin
> > >>> <
> > >>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> > >> Metaspace
> > >>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>> repeated
> > >>>>>>>>>>>>>>>>>>> job”
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
> > >>>> covered
> > >>>>>>>>>> by:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
> > >>> be
> > >>>>>>>>>> handled as
> > >>>>>>>>>>>>> Fatal
> > >>>>>>>>>>>>>>>> Error
> > >>>>>>>>>>>>>>>>>>> in TaskManager
> > >>>>>>>>>>>>>>>>>>> no full consensus there but improving error
> > >>>>>> message
> > >>>>>>>>>> for
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> task
> > >>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
> > >>>>>>>>>> Metaspace to
> > >>>>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>>>> see further
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
> > >>> "SdkMBeanRegistrySupport"
> > >>>>>> from
> > >>>>>>>>>>>>> dynamically
> > >>>>>>>>>>>>>>>>> loaded
> > >>>>>>>>>>>>>>>>>>> AWS connectors
> > >>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
> > >>>> close
> > >>>>>> to
> > >>>>>>>>>> be
> > >>>>>>>>>>>>> resolved
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>>> Metaspace"
> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> > >>>> bugfix
> > >>>>>>>>>> release?
> > >>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>> changes
> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> > >>>>>> memory
> > >>>>>>>>>> on
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> setups
> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
> > >>>>>> resolved
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> reported
> > >>>>>>>>>>>>>>>>> problems
> > >>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
> > >>> merged
> > >>>>>> it
> > >>>>>>>>>> today
> > >>>>>>>>>>> as
> > >>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>> were
> > >>>>>>>>>>>>>>>>>>> no more concerns.
> > >>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
> > >> message
> > >>>>>> error
> > >>>>>>>>>> with
> > >>>>>>>>>>>>>>> reference
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> the metaspace option should help the most
> > >>>>>>>>>>>>>>>>>>> because user would not have to read all docs
> > >> to
> > >>>> fix
> > >>>>>> it
> > >>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>> Andrey
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> > >>>>>>>>>> sewen@apache.org>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> > >> Metaspace
> > >>>> OOM
> > >>>>>>>>>> error
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>>>> repeated
> > >>>>>>>>>>>>>>>>>>> job"
> > >>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
> > >> exact
> > >>>>>>>>>> solution,
> > >>>>>>>>>>> yet,
> > >>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> some
> > >>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
> > >>> are
> > >>>>>>>>>> hard to
> > >>>>>>>>>>>>>> predict,
> > >>>>>>>>>>>>>>>> so I
> > >>>>>>>>>>>>>>>>>>> am
> > >>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>>> Metaspace"
> > >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> > >>>> bugfix
> > >>>>>>>>>> release?
> > >>>>>>>>>>>> It
> > >>>>>>>>>>>>>>>> changes
> > >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> > >>>>>> memory
> > >>>>>>>>>> on
> > >>>>>>>>>>>>> existing
> > >>>>>>>>>>>>>>>> setups
> > >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
> > >>> two
> > >>>>>>>>>> issues
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>> think
> > >>>>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> > >>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
> > >> eye
> > >>>> on
> > >>>>>>>>>> these
> > >>>>>>>>>>> two.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> > >>>>>>>>>> imjark@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
> > >> should
> > >>>> also
> > >>>>>>>>>> try to
> > >>>>>>>>>>>> get
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> following issues:
> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
> > >>>> flink-conf
> > >>>>>>>>>>>> parameters
> > >>>>>>>>>>>>>> from
> > >>>>>>>>>>>>>>>> SQL
> > >>>>>>>>>>>>>>>>>>> CLI
> > >>>>>>>>>>>>>>>>>>>>>> environment
> > >>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
> > >>>>>>>>>> watermark
> > >>>>>>>>>>>>> interval,
> > >>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
> > >>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
> > >>>>>> uncatched
> > >>>>>>>>>>>>> exception
> > >>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>>> thrown
> > >>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
> > >> using
> > >>>> SQL
> > >>>>>> CLI
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>> Jark
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> > >>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
> > >> added
> > >>>>>>>>>> 1.10.1 into
> > >>>>>>>>>>>>>>>> FLINK-16225
> > >>>>>>>>>>>>>>>>>>>>> fix
> > >>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
> > >>>> Critical.
> > >>>>>>>>>> Will
> > >>>>>>>>>>> also
> > >>>>>>>>>>>>>> watch
> > >>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> > >>> Zagrebin
> > >>>> <
> > >>>>>>>>>>>>>>>> azagrebin@apache.org
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
> > >>>>>>>>>> discussion!
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Apart from
> > >>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
> > >>> JVM
> > >>>>>>>>>> Metaspace
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
> > >>>>>> following
> > >>>>>>>>>>>> issues:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> > >>>> should
> > >>>>>> be
> > >>>>>>>>>>> handled
> > >>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> Fatal
> > >>>>>>>>>>>>>>>>>>>>>> Error
> > >>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>> TaskManager
> > >>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
> > >>> even
> > >>>>>> in a
> > >>>>>>>>>>> better
> > >>>>>>>>>>>>> way
> > >>>>>>>>>>>>>>>>> because
> > >>>>>>>>>>>>>>>>>>>>>> OOM
> > >>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
> > >>>>>> immediately
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
> > >> loader
> > >>>> to
> > >>>>>>>>>> lifetime
> > >>>>>>>>>>>> of
> > >>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>> slot
> > >>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
> > >>> against
> > >>>>>> class
> > >>>>>>>>>>>> loading
> > >>>>>>>>>>>>>>> leaks
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
> > >>> when
> > >>>>>>>>>> submitting
> > >>>>>>>>>>>>> batch
> > >>>>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>>>>>>>> (instead
> > >>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
> > >>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
> > >>>> multiple
> > >>>>>>>>>> users
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>> Andrey
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> > >>> Li <
> > >>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
> > >>> the
> > >>>>>>>>>> 1.10.1
> > >>>>>>>>>>>>> release.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
> > >> are
> > >>>>>> looking
> > >>>>>>>>>>>> forward
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> them.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> > >>> Guo <
> > >>>>>>>>>>>>>>> karmagyz@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
> > >> cycle.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> > >>>> Song
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
> > >>> get
> > >>>>>> some
> > >>>>>>>>>>> inputs
> > >>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>> prioritizing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
> > >>> good
> > >>>> to
> > >>>>>>>>>> me.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
> > >> Li <
> > >>>>>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
> > >>> patch
> > >>>>>>>>>> release
> > >>>>>>>>>>> of
> > >>>>>>>>>>>> a
> > >>>>>>>>>>>>>>> major
> > >>>>>>>>>>>>>>>>>>>>>>>> version
> > >>>>>>>>>>>>>>>>>>>>>>>>>> will
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
> > >>>>>>>>>> depending on
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> speed
> > >>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>> blocker
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
> > >> 28th
> > >>>>>> [1],
> > >>>>>>>>>>> released
> > >>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>> Jul
> > >>>>>>>>>>>>>>>>>>>>>> 3rd
> > >>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
> > >> 23rd
> > >>>>>> [3],
> > >>>>>>>>>>> released
> > >>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>> Oct
> > >>>>>>>>>>>>>>>>>>>>>> 19th
> > >>>>>>>>>>>>>>>>>>>>>>>> [4]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
> > >>>>>> course,
> > >>>>>>>>>> but
> > >>>>>>>>>>>> could
> > >>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>> as a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
> > >> let
> > >>> me
> > >>>>>>>>>> know if
> > >>>>>>>>>>> any
> > >>>>>>>>>>>>>>>>>>>>> concerns
> > >>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> > >>>> Song
> > >>>>>> <
> > >>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
> > >>>>>> volunteering
> > >>>>>>>>>> to be
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>> manager.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
> > >>>> plan
> > >>>>>> for
> > >>>>>>>>>>>> 1.10.1?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> > >>>> Zhijiang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
> > >> .invalid>
> > >>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
> > >> Yu!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>> ------------------------------------------------------------------
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
> > >> Flink
> > >>>>>> 1.10.1
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
> > >> Yes
> > >>>> will
> > >>>>>>>>>> also
> > >>>>>>>>>>> keep
> > >>>>>>>>>>>>> an
> > >>>>>>>>>>>>>>>>>>>>> eye
> > >>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> > >>>>>> Cheng <
> > >>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
> > >>>> discussion
> > >>>>>> and
> > >>>>>>>>>>>>> volunteer
> > >>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
> > >>> issues[1]
> > >>>>>>>>>> which are
> > >>>>>>>>>>>>>> marked
> > >>>>>>>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> > >>>> year
> > >>>>>> in
> > >>>>>>>>>>> NOTICE
> > >>>>>>>>>>>>>> files
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> > >>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> > >>>>>>>>>> and
> > >>>>>>>>>>>> usrlib
> > >>>>>>>>>>>>>>>>>>>>>>> directory
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
> > >> SearchTemplateRequest
> > >>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
> > >>>>>>>>>>>>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>> use
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
> > >>> reporting
> > >>>>>> when
> > >>>>>>>>>>>>> submitting
> > >>>>>>>>>>>>>>>>>>>>>> batch
> > >>>>>>>>>>>>>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
> > >>> in
> > >>>>>>>>>> 1.10.1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> > >>> Li
> > >>>> <
> > >>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
> > >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
> > >>> helpful.
> > >>>>>>>>>> Thanks a
> > >>>>>>>>>>>> lot!
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> > >>>> jincheng
> > >>>>>>>>>> sun <
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
> > >> discussion
> > >>>>>> Yu. I
> > >>>>>>>>>> would
> > >>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>> give
> > >>>>>>>>>>>>>>>>>>>>>>>>>> you a
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
> > >>>>>>>>>> finished.(If you
> > >>>>>>>>>>>>>>>>>>>>> need)
> > >>>>>>>>>>>>>>>>>>>>>>> :)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jincheng
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
> > >>>>>> 于2020年3月10日周二
> > >>>>>>>>>>>> 下午5:49写道：
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It has been almost one month
> > >>> since
> > >>>> we
> > >>>>>>>>>>> released
> > >>>>>>>>>>>>>>>>>>>>>> Flink
> > >>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.0. We
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have more than 40 resolved
> > >>>>>>>>>> improvements/bugs
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.10
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and I propose to start the
> > >> 1.10.1
> > >>>>>>>>>> release
> > >>>>>>>>>>>> cycle.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove
> > >> the
> > >>>>>>>>>> license and
> > >>>>>>>>>>>>>>>>>>>>> notice
> > >>>>>>>>>>>>>>>>>>>>>>>> file
> > >>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-ml-lib
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> module
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB
> > >>> resource
> > >>>>>> leak
> > >>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-state-processor-api
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
> > >>>> should
> > >>>>>> be
> > >>>>>>>>>> known
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>> HiveCatalog
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-2336
> > >>>>>>>>>> ArrayIndexOufOBoundsException in
> > >>>>>>>>>>>>>>>>>>>>>>>>> TypeExtractor
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapping
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample
> > >> is
> > >>>>>>>>>> failed if
> > >>>>>>>>>>>>>>>>>>>>> running
> > >>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>> blink
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> planner
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
> > >>>> constraints
> > >>>>>>>>>> are not
> > >>>>>>>>>>>>>>>>>>>>> reset
> > >>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>> task
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recovery
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DefaultScheduler
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> > >>>>>> function
> > >>>>>>>>>> using
> > >>>>>>>>>>>> sql
> > >>>>>>>>>>>>>>>>>>>>>>>> casuing
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ValidationException: SQL
> > >>> validation
> > >>>>>>>>>> failed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Furthermore, I think the
> > >>> following
> > >>>>>>>>>> issues
> > >>>>>>>>>>>> should
> > >>>>>>>>>>>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>>>>>>>> merged
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release (especially the
> > >> Metaspace
> > >>>> OOM
> > >>>>>>>>>> issue):
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak
> > >> causes
> > >>>>>>>>>> Metaspace
> > >>>>>>>>>>> OOM
> > >>>>>>>>>>>>>>>>>>>>>> error
> > >>>>>>>>>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> repeated
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submission
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
> > >>>> value
> > >>>>>>>>>> for JVM
> > >>>>>>>>>>>>>>>>>>>>>>> Metaspace
> > >>>>>>>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
> > >>>> produces
> > >>>>>>>>>> wrong
> > >>>>>>>>>>>>>>>>>>>>>> aggregate
> > >>>>>>>>>>>>>>>>>>>>>>>>>> results
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clean up
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can
> > >>> not
> > >>>>>>>>>> extract
> > >>>>>>>>>>>>>>>>>>>>> correct
> > >>>>>>>>>>>>>>>>>>>>>>>>> unique
> > >>>>>>>>>>>>>>>>>>>>>>>>>> key
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would volunteer as the
> > >> release
> > >>>>>>>>>> manager and
> > >>>>>>>>>>>> kick
> > >>>>>>>>>>>>>>>>>>>>>> off
> > >>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> process
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> once blocker issues are merged.
> > >>>> What
> > >>>>>> do
> > >>>>>>>>>> you
> > >>>>>>>>>>>>>>>>>>>>> think?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If there are any concerns or
> > >>>> missing
> > >>>>>>>>>> blocker
> > >>>>>>>>>>>>>>>>>>>>> issues
> > >>>>>>>>>>>>>>>>>>>>>>>> need
> > >>>>>>>>>>>>>>>>>>>>>>>>>> to be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixed
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1, please let me know.
> > >>> Thanks.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>>>>>> Best, Jingsong Lee
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Hequn Cheng <he...@apache.org>.
Thanks a lot for your great work Yu! Looking forward to the RC.
Best, Hequn
On Thu, Apr 16, 2020 at 10:35 AM Dian Fu <di...@gmail.com> wrote:
> Thanks a lot for driving this, Yu! Looking forward for the first RC of
> 1.10.1.
>
> > 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
> >
> > Looking forward the first RC of Flink 1.10.1 .
> > Good job Yu!
> >
> > Best,
> > Jincheng
> >
> >
> >
> > Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
> >
> >> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> >> released.
> >>
> >> Best,
> >> Jark
> >>
> >> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org>
> wrote:
> >>
> >>> Great to see that will have the first RC for Flink 1.10.1 soon. Thanks
> a
> >>> lot for driving this effort Yu!
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> >>>
> >>>> Thanks Weike and all others for the efforts!
> >>>>
> >>>> Here comes the latest status, we are in good shape and plan to produce
> >>> RC1
> >>>> next week.
> >>>>
> >>>> * Blockers (1 left)
> >>>> - [Closed] FLINK-16018 Improve error reporting when submitting batch
> >>> job
> >>>> (instead of AskTimeoutException)
> >>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>> repeated
> >>>> job submission
> >>>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> >>> when
> >>>> use flink-sql-connector-elasticsearch7
> >>>> - [Closed] FLINK-16262 Class loader problem with
> >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >>>> minimise its OutOfMemoryError
> >>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> >>> client
> >>>> can retrieve JobResult
> >>>> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> >>>> throws UnsupportedOperationException
> >>>> - [Closed] FLINK-16626 Exception encountered when cancelling a job in
> >>>> yarn per-job mode
> >>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
> >> the
> >>>> input column is of composite type
> >>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> >> memory
> >>>> state backends
> >>>>
> >>>> * Critical (1 left)
> >>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> >>>> with state clean up
> >>>> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> >> key
> >>>> for UpsertStreamTableSink
> >>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
> >>> be
> >>>> handled as Fatal Error in TaskManager
> >>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> >> error
> >>>> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> >>> of a
> >>>> slot
> >>>>
> >>>> Please let me know if any concerns/comments. Thanks.
> >>>>
> >>>> Best Regards,
> >>>> Yu
> >>>>
> >>>>
> >>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> >>> wrote:
> >>>>
> >>>>> Hi Yu,
> >>>>>
> >>>>> Thanks for your updates. I am still working on the fix for
> >> FLINK-16626
> >>>> and
> >>>>> it is expected to be completed by this Sunday after thorough testing.
> >>>>>
> >>>>> Sincerely,
> >>>>> Weike
> >>>>>
> >>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> >>>>>
> >>>>>> Updates for 1.10.1 watched issues (we are in good progress and
> >> almost
> >>>>>> there
> >>>>>> to produce the first RC, thanks all for the efforts):
> >>>>>>
> >>>>>> * Blockers (3 left)
> >>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
> >> batch
> >>>> job
> >>>>>> (instead of AskTimeoutException)
> >>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>>>>> repeated
> >>>>>> job submission
> >>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> >> ClassNotFoundException
> >>>> when
> >>>>>> use flink-sql-connector-elasticsearch7
> >>>>>> - [Closed] FLINK-16262 Class loader problem with
> >>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >>>>>> minimise its OutOfMemoryError
> >>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> >>> memory
> >>>>>> state backends
> >>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
> >>>> cancelling a
> >>>>>> job in yarn per-job mode
> >>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> >>>>>> client
> >>>>>> can retrieve JobResult
> >>>>>> - [PR reviewed] FLINK-16913
> >>>> ReadableConfigToConfigurationAdapter#getEnum
> >>>>>> throws UnsupportedOperationException
> >>>>>>
> >>>>>> * Critical (1 left)
> >>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> >>> error
> >>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>> results
> >>>>>> with state clean up
> >>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
> >> unique
> >>>> key
> >>>>>> for UpsertStreamTableSink
> >>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> >>> should
> >>>> be
> >>>>>> handled as Fatal Error in TaskManager
> >>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
> >> lifetime
> >>>> of
> >>>>>> a
> >>>>>> slot
> >>>>>>
> >>>>>> Please let me know if you find any missing ones, thanks.
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Yu
> >>>>>>
> >>>>>>
> >>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
> >>>>>>>
> >>>>>>> * Blockers (4 left)
> >>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> >>>>>> submitting
> >>>>>>> batch job (instead of AskTimeoutException)
> >>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >>>>>>> repeated job submission
> >>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
> >>> ClassNotFoundException
> >>>>>> when
> >>>>>>> use flink-sql-connector-elasticsearch7
> >>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
> >>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> >> to
> >>>>>>> minimise its OutOfMemoryError
> >>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
> >>>> memory
> >>>>>>> state backends
> >>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> >>>>>> MiniCluster
> >>>>>>> before client can retrieve JobResult
> >>>>>>>
> >>>>>>> * Critical (2 left)
> >>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>>> results
> >>>>>>> with state clean up
> >>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> >> correct
> >>>>>>> unique key for UpsertStreamTableSink
> >>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
> >> should
> >>>> be
> >>>>>>> handled as Fatal Error in TaskManager
> >>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
> >> a
> >>>> slot
> >>>>>>>
> >>>>>>> There's a new blocker added. Please let me know if you find any
> >>> other
> >>>>>>> issues should be put into the watch list. Thanks.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Yu
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> Here is the status update of issues in 1.10.1 watch list:
> >>>>>>>>
> >>>>>>>> * Blockers (7)
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
> >>>>>>>> submitting batch job (instead of AskTimeoutException)
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> >> OOM
> >>>>>> error
> >>>>>>>> on repeated job submission
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >>>>>>>> ClassNotFoundException when use
> >> flink-sql-connector-elasticsearch7
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> >>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> >>> to
> >>>>>>>> minimise its OutOfMemoryError
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
> >> files
> >>>>>>>>
> >>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with memory
> >>>> state
> >>>>>>>> backends
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> * Critical (4)
> >>>>>>>>
> >>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> >>>> results
> >>>>>>>> with state clean up
> >>>>>>>>
> >>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
> >>> correct
> >>>>>>>> unique key for UpsertStreamTableSink
> >>>>>>>>
> >>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> >>> be
> >>>>>>>> handled as Fatal Error in TaskManager
> >>>>>>>>
> >>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
> >> of a
> >>>>>> slot
> >>>>>>>>
> >>>>>>>> Please let me know if you see any new blockers to add to the
> >> list.
> >>>>>> Thanks.
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Yu
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks for the updates Till!
> >>>>>>>>>
> >>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
> >> and
> >>>>>>>>> complete fix separately, and only include the easy one in
> >> 1.10.1?
> >>> Or
> >>>>>> please
> >>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
> >>>>>> nothing"
> >>>>>>>>> policy is preferred (smile). Thanks.
> >>>>>>>>>
> >>>>>>>>> Best Regards,
> >>>>>>>>> Yu
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
> >> trohrmann@apache.org
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
> >> our
> >>>>>> release
> >>>>>>>>>> manager Yu.
> >>>>>>>>>>
> >>>>>>>>>> I think we can soon merge the increase of metaspace size and
> >>>>>> improving
> >>>>>>>>>> the
> >>>>>>>>>> error message. The assumption is that we currently don't have
> >> too
> >>>>>> many
> >>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> >>> course,
> >>>>>> the
> >>>>>>>>>> sooner we release the bug fix release, the fewer deployments
> >> will
> >>>> be
> >>>>>>>>>> affected by this change.
> >>>>>>>>>>
> >>>>>>>>>> For FLINK-16018, I think there would be an easy solution which
> >> we
> >>>>>> could
> >>>>>>>>>> include in the bug fix release. The proper fix will most likely
> >>>> take
> >>>>>> a
> >>>>>>>>>> bit
> >>>>>>>>>> longer, though.
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Till
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> >>>>>> azagrebin@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> >> user
> >>>>>>>>>> mailing
> >>>>>>>>>>> list
> >>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> >>>> Specifically
> >>>>>>>>>> asking
> >>>>>>>>>>>> for who has very small TM setups?
> >>>>>>>>>>>
> >>>>>>>>>>> There has been a survey about this topic since 10 days:
> >>>>>>>>>>>
> >>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
> >> 1.10`
> >>>>>>>>>>> I can ask there about the small TM setups.
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> >>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
> >>> on
> >>>>>>>>>> restore
> >>>>>>>>>>> with
> >>>>>>>>>>>> memory state backends
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
> >>>>>> There're
> >>>>>>>>>>> totally
> >>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
> >>>>>> which 14
> >>>>>>>>>>>> already resolved and 31 left, and the below ones are
> >> watched
> >>>>>>>>>> (meaning
> >>>>>>>>>>> that
> >>>>>>>>>>>> once the below ones got fixed, we will kick of the
> >> releasing
> >>>> with
> >>>>>>>>>> left
> >>>>>>>>>>> ones
> >>>>>>>>>>>> postponed to 1.10.2 unless objections):
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Blockers (7)
> >>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
> >>> when
> >>>>>>>>>>> submitting
> >>>>>>>>>>>> batch job (instead of AskTimeoutException)
> >>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
> >>> Metaspace
> >>>>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>>> on repeated job submission
> >>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >>>>>>>>>>> ClassNotFoundException
> >>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
> >>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
> >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> >> directory
> >>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
> >>>>>> Metaspace to
> >>>>>>>>>>>> minimise its OutOfMemoryError
> >>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
> >>>> files
> >>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
> >>>> memory
> >>>>>>>>>> state
> >>>>>>>>>>>> backends
> >>>>>>>>>>>>
> >>>>>>>>>>>> * Critical (4)
> >>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
> >> aggregate
> >>>>>>>>>> results
> >>>>>>>>>>> with
> >>>>>>>>>>>> state clean up
> >>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
> >> extract
> >>>>>> correct
> >>>>>>>>>>> unique
> >>>>>>>>>>>> key for UpsertStreamTableSink
> >>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> >>>>>> should be
> >>>>>>>>>>>> handled as Fatal Error in TaskManager
> >>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
> >>> lifetime
> >>>>>> of a
> >>>>>>>>>> slot
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please raise your hand if you find any other issues should
> >> be
> >>>> put
> >>>>>>>>>> into
> >>>>>>>>>>> this
> >>>>>>>>>>>> list, thanks.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please also note that 1.10.2 version is already created
> >>> (thanks
> >>>>>> for
> >>>>>>>>>> the
> >>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
> >>> to
> >>>> it
> >>>>>>>>>> when
> >>>>>>>>>>>> necessary.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Yu
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
> >> sewen@apache.org
> >>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
> >>> user
> >>>>>>>>>> mailing
> >>>>>>>>>>>> list
> >>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
> >>>>>> Specifically
> >>>>>>>>>>> asking
> >>>>>>>>>>>>> for who has very small TM setups?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the reminder Stephan and the
> >> inputs/discussion
> >>>>>>>>>> Andrey and
> >>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
> >>>>>>>>>> introduce new
> >>>>>>>>>>>>>> issues when resolving the existing one. Will watch
> >>> closely
> >>>>>>>>>> about the
> >>>>>>>>>>>>>> discussion and conclusion around metaspace
> >> configuration.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
> >>> optimistic
> >>>>>> that
> >>>>>>>>>>>>> reserving
> >>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
> >>> cause a
> >>>>>> big
> >>>>>>>>>>>> problem,
> >>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
> >>>>>> memory
> >>>>>>>>>> per TM
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>> the common case.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> >>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
> >>>>>> default
> >>>>>>>>>> value
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
> >>>> default
> >>>>>>>>>>>>>>> 'flink-conf.yaml'.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> That means:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
> >>> it's
> >>>>>> not
> >>>>>>>>>>> really
> >>>>>>>>>>>>>> taking
> >>>>>>>>>>>>>>> memory from heap / managed for metaspace.
> >>>>>>>>>>>>>>> - For people using custom configuration files:
> >>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
> >>>> metaspace
> >>>>>>>>>> does
> >>>>>>>>>>> not
> >>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>> affect the flink memory internal components
> >>> (heap,
> >>>>>>>>>> managed,
> >>>>>>>>>>>>> etc.).
> >>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
> >>>> likely
> >>>>>>>>>> active
> >>>>>>>>>>>>>> setups),
> >>>>>>>>>>>>>>> then they will have less heap / managed memory,
> >>> or
> >>>>>>>>>> worse,
> >>>>>>>>>>> their
> >>>>>>>>>>>>>>> configuration may not work without adjustment.
> >>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
> >> 4
> >>> GB
> >>>>>> or
> >>>>>>>>>> more),
> >>>>>>>>>>>> it
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
> >>> more
> >>>>>>>>>> memory.
> >>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
> >>> GB)
> >>>>>> the
> >>>>>>>>>>>> metaspace
> >>>>>>>>>>>>>>> increase is more likely to cause problem.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So basically only people have small 'process.size' in
> >>>>>> custom
> >>>>>>>>>> config
> >>>>>>>>>>>>> file
> >>>>>>>>>>>>>>> are really affected. I'm not sure what is the
> >>> proportion
> >>>> of
> >>>>>>>>>> such
> >>>>>>>>>>> use
> >>>>>>>>>>>>>> cases
> >>>>>>>>>>>>>>> though. (From questions asked on the user ML,
> >> probably
> >>>> not
> >>>>>>>>>> much).
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
> >> should
> >>>> not
> >>>>>> go
> >>>>>>>>>> into
> >>>>>>>>>>>>>> 1.10.1,
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We have to trade off the fact that we may break
> >> some
> >>>>>>>>>> deployments
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
> >>>>>>>>>>>>>>>> I simply lack the data points / insight at the
> >> moment
> >>>> to
> >>>>>>>>>>> understand
> >>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>> significant both cases are, meaning how many users
> >>>> would
> >>>>>> be
> >>>>>>>>>>>> affected
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> how badly.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Independent of that, improving the error message is
> >>>>>> always
> >>>>>>>>>>> helpful.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> >>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> >> because
> >>>>>> users
> >>>>>>>>>> expect
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> upgrade
> >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> >>> be
> >>>>>>>>>> possible
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> change.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
> >> only
> >>>>>> try to
> >>>>>>>>>>>> improve
> >>>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>> message and docs.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
> >>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
> >>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
> >> because
> >>>>>> users
> >>>>>>>>>> expect
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> upgrade
> >>>>>>>>>>>>>>>>>> that without config adjustments. That might not
> >>> be
> >>>>>>>>>> possible
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>> change.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
> >> Zagrebin
> >>> <
> >>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> >> Metaspace
> >>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>> repeated
> >>>>>>>>>>>>>>>>>>> job”
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
> >>>> covered
> >>>>>>>>>> by:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
> >>> be
> >>>>>>>>>> handled as
> >>>>>>>>>>>>> Fatal
> >>>>>>>>>>>>>>>> Error
> >>>>>>>>>>>>>>>>>>> in TaskManager
> >>>>>>>>>>>>>>>>>>> no full consensus there but improving error
> >>>>>> message
> >>>>>>>>>> for
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> task
> >>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
> >>>>>>>>>> Metaspace to
> >>>>>>>>>>>>>> minimise
> >>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>>>> see further
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
> >>> "SdkMBeanRegistrySupport"
> >>>>>> from
> >>>>>>>>>>>>> dynamically
> >>>>>>>>>>>>>>>>> loaded
> >>>>>>>>>>>>>>>>>>> AWS connectors
> >>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
> >>>> close
> >>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>>> resolved
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>>> Metaspace"
> >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> >>>> bugfix
> >>>>>>>>>> release?
> >>>>>>>>>>>> It
> >>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> >>>>>> memory
> >>>>>>>>>> on
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> setups
> >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
> >>>>>> resolved
> >>>>>>>>>> the
> >>>>>>>>>>>>> reported
> >>>>>>>>>>>>>>>>> problems
> >>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
> >>> merged
> >>>>>> it
> >>>>>>>>>> today
> >>>>>>>>>>> as
> >>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> were
> >>>>>>>>>>>>>>>>>>> no more concerns.
> >>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
> >> message
> >>>>>> error
> >>>>>>>>>> with
> >>>>>>>>>>>>>>> reference
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> the metaspace option should help the most
> >>>>>>>>>>>>>>>>>>> because user would not have to read all docs
> >> to
> >>>> fix
> >>>>>> it
> >>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>> Andrey
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> >>>>>>>>>> sewen@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
> >> Metaspace
> >>>> OOM
> >>>>>>>>>> error
> >>>>>>>>>>> on
> >>>>>>>>>>>>>>> repeated
> >>>>>>>>>>>>>>>>>>> job"
> >>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
> >> exact
> >>>>>>>>>> solution,
> >>>>>>>>>>> yet,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> some
> >>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
> >>> are
> >>>>>>>>>> hard to
> >>>>>>>>>>>>>> predict,
> >>>>>>>>>>>>>>>> so I
> >>>>>>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>>> Metaspace"
> >>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
> >>>> bugfix
> >>>>>>>>>> release?
> >>>>>>>>>>>> It
> >>>>>>>>>>>>>>>> changes
> >>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
> >>>>>> memory
> >>>>>>>>>> on
> >>>>>>>>>>>>> existing
> >>>>>>>>>>>>>>>> setups
> >>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
> >>> two
> >>>>>>>>>> issues
> >>>>>>>>>>> and
> >>>>>>>>>>>>>> think
> >>>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> >>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
> >> eye
> >>>> on
> >>>>>>>>>> these
> >>>>>>>>>>> two.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> >>>>>>>>>> imjark@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
> >> should
> >>>> also
> >>>>>>>>>> try to
> >>>>>>>>>>>> get
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> following issues:
> >>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
> >>>> flink-conf
> >>>>>>>>>>>> parameters
> >>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>> SQL
> >>>>>>>>>>>>>>>>>>> CLI
> >>>>>>>>>>>>>>>>>>>>>> environment
> >>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
> >>>>>>>>>> watermark
> >>>>>>>>>>>>> interval,
> >>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
> >>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
> >>>>>> uncatched
> >>>>>>>>>>>>> exception
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> thrown
> >>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
> >> using
> >>>> SQL
> >>>>>> CLI
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>> Jark
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> >>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
> >> added
> >>>>>>>>>> 1.10.1 into
> >>>>>>>>>>>>>>>> FLINK-16225
> >>>>>>>>>>>>>>>>>>>>> fix
> >>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
> >>>> Critical.
> >>>>>>>>>> Will
> >>>>>>>>>>> also
> >>>>>>>>>>>>>> watch
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> >>> Zagrebin
> >>>> <
> >>>>>>>>>>>>>>>> azagrebin@apache.org
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
> >>>>>>>>>> discussion!
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Apart from
> >>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
> >>> JVM
> >>>>>>>>>> Metaspace
> >>>>>>>>>>> to
> >>>>>>>>>>>>>>>> minimise
> >>>>>>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
> >>>>>> following
> >>>>>>>>>>>> issues:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> >>>> should
> >>>>>> be
> >>>>>>>>>>> handled
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> Fatal
> >>>>>>>>>>>>>>>>>>>>>> Error
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>> TaskManager
> >>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
> >>> even
> >>>>>> in a
> >>>>>>>>>>> better
> >>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>> because
> >>>>>>>>>>>>>>>>>>>>>> OOM
> >>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
> >>>>>> immediately
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
> >> loader
> >>>> to
> >>>>>>>>>> lifetime
> >>>>>>>>>>>> of
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> slot
> >>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
> >>> against
> >>>>>> class
> >>>>>>>>>>>> loading
> >>>>>>>>>>>>>>> leaks
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
> >>> when
> >>>>>>>>>> submitting
> >>>>>>>>>>>>> batch
> >>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>> (instead
> >>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
> >>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
> >>>> multiple
> >>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>> Andrey
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> >>> Li <
> >>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
> >>> the
> >>>>>>>>>> 1.10.1
> >>>>>>>>>>>>> release.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
> >> are
> >>>>>> looking
> >>>>>>>>>>>> forward
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> them.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> >>> Guo <
> >>>>>>>>>>>>>>> karmagyz@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
> >> cycle.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> >>>> Song
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
> >>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
> >>> get
> >>>>>> some
> >>>>>>>>>>> inputs
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>> prioritizing
> >>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
> >>> good
> >>>> to
> >>>>>>>>>> me.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
> >> Li <
> >>>>>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
> >>> patch
> >>>>>>>>>> release
> >>>>>>>>>>> of
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>>> major
> >>>>>>>>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
> >>>>>>>>>> depending on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> speed
> >>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>> blocker
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
> >> 28th
> >>>>>> [1],
> >>>>>>>>>>> released
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> Jul
> >>>>>>>>>>>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
> >> 23rd
> >>>>>> [3],
> >>>>>>>>>>> released
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>>>> Oct
> >>>>>>>>>>>>>>>>>>>>>> 19th
> >>>>>>>>>>>>>>>>>>>>>>>> [4]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
> >>>>>> course,
> >>>>>>>>>> but
> >>>>>>>>>>>> could
> >>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>> as a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
> >> let
> >>> me
> >>>>>>>>>> know if
> >>>>>>>>>>> any
> >>>>>>>>>>>>>>>>>>>>> concerns
> >>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> >>>> Song
> >>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
> >>>>>> volunteering
> >>>>>>>>>> to be
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>> manager.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
> >>>> plan
> >>>>>> for
> >>>>>>>>>>>> 1.10.1?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> >>>> Zhijiang
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
> >> .invalid>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
> >> Yu!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>> ------------------------------------------------------------------
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
> >> Flink
> >>>>>> 1.10.1
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
> >> Yes
> >>>> will
> >>>>>>>>>> also
> >>>>>>>>>>> keep
> >>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>>>>> eye
> >>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> >>>>>> Cheng <
> >>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
> >>>> discussion
> >>>>>> and
> >>>>>>>>>>>>> volunteer
> >>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
> >>> issues[1]
> >>>>>>>>>> which are
> >>>>>>>>>>>>>> marked
> >>>>>>>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> >>>> year
> >>>>>> in
> >>>>>>>>>>> NOTICE
> >>>>>>>>>>>>>> files
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> >>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> >>>>>>>>>> and
> >>>>>>>>>>>> usrlib
> >>>>>>>>>>>>>>>>>>>>>>> directory
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
> >> SearchTemplateRequest
> >>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
> >>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
> >>> reporting
> >>>>>> when
> >>>>>>>>>>>>> submitting
> >>>>>>>>>>>>>>>>>>>>>> batch
> >>>>>>>>>>>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
> >>> in
> >>>>>>>>>> 1.10.1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>
> >> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> >>> Li
> >>>> <
> >>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
> >>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
> >>> helpful.
> >>>>>>>>>> Thanks a
> >>>>>>>>>>>> lot!
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> >>>> jincheng
> >>>>>>>>>> sun <
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
> >> discussion
> >>>>>> Yu. I
> >>>>>>>>>> would
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>> give
> >>>>>>>>>>>>>>>>>>>>>>>>>> you a
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
> >>>>>>>>>> finished.(If you
> >>>>>>>>>>>>>>>>>>>>> need)
> >>>>>>>>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jincheng
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
> >>>>>> 于2020年3月10日周二
> >>>>>>>>>>>> 下午5:49写道：
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It has been almost one month
> >>> since
> >>>> we
> >>>>>>>>>>> released
> >>>>>>>>>>>>>>>>>>>>>> Flink
> >>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.0. We
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have more than 40 resolved
> >>>>>>>>>> improvements/bugs
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.10
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and I propose to start the
> >> 1.10.1
> >>>>>>>>>> release
> >>>>>>>>>>>> cycle.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Most noticeable fixes are:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove
> >> the
> >>>>>>>>>> license and
> >>>>>>>>>>>>>>>>>>>>> notice
> >>>>>>>>>>>>>>>>>>>>>>>> file
> >>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-ml-lib
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> module
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB
> >>> resource
> >>>>>> leak
> >>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-state-processor-api
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
> >>>> should
> >>>>>> be
> >>>>>>>>>> known
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> HiveCatalog
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-2336
> >>>>>>>>>> ArrayIndexOufOBoundsException in
> >>>>>>>>>>>>>>>>>>>>>>>>> TypeExtractor
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> when
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapping
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample
> >> is
> >>>>>>>>>> failed if
> >>>>>>>>>>>>>>>>>>>>> running
> >>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>> blink
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> planner
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
> >>>> constraints
> >>>>>>>>>> are not
> >>>>>>>>>>>>>>>>>>>>> reset
> >>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>> task
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recovery
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DefaultScheduler
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> >>>>>> function
> >>>>>>>>>> using
> >>>>>>>>>>>> sql
> >>>>>>>>>>>>>>>>>>>>>>>> casuing
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ValidationException: SQL
> >>> validation
> >>>>>>>>>> failed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Furthermore, I think the
> >>> following
> >>>>>>>>>> issues
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>>>>>>>> merged
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> before
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release (especially the
> >> Metaspace
> >>>> OOM
> >>>>>>>>>> issue):
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak
> >> causes
> >>>>>>>>>> Metaspace
> >>>>>>>>>>> OOM
> >>>>>>>>>>>>>>>>>>>>>> error
> >>>>>>>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> repeated
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submission
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
> >>>> value
> >>>>>>>>>> for JVM
> >>>>>>>>>>>>>>>>>>>>>>> Metaspace
> >>>>>>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> minimise
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
> >>>> produces
> >>>>>>>>>> wrong
> >>>>>>>>>>>>>>>>>>>>>> aggregate
> >>>>>>>>>>>>>>>>>>>>>>>>>> results
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clean up
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can
> >>> not
> >>>>>>>>>> extract
> >>>>>>>>>>>>>>>>>>>>> correct
> >>>>>>>>>>>>>>>>>>>>>>>>> unique
> >>>>>>>>>>>>>>>>>>>>>>>>>> key
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UpsertStreamTableSink
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would volunteer as the
> >> release
> >>>>>>>>>> manager and
> >>>>>>>>>>>> kick
> >>>>>>>>>>>>>>>>>>>>>> off
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> process
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> once blocker issues are merged.
> >>>> What
> >>>>>> do
> >>>>>>>>>> you
> >>>>>>>>>>>>>>>>>>>>> think?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If there are any concerns or
> >>>> missing
> >>>>>>>>>> blocker
> >>>>>>>>>>>>>>>>>>>>> issues
> >>>>>>>>>>>>>>>>>>>>>>>> need
> >>>>>>>>>>>>>>>>>>>>>>>>>> to be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1, please let me know.
> >>> Thanks.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>>>>>> Best, Jingsong Lee
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Dian Fu <di...@gmail.com>.
Thanks a lot for driving this, Yu! Looking forward for the first RC of 1.10.1.
> 在 2020年4月16日，上午10:24，jincheng sun <su...@gmail.com> 写道：
>
> Looking forward the first RC of Flink 1.10.1 .
> Good job Yu!
>
> Best,
> Jincheng
>
>
>
> Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
>
>> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
>> released.
>>
>> Best,
>> Jark
>>
>> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org> wrote:
>>
>>> Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
>>> lot for driving this effort Yu!
>>>
>>> Cheers,
>>> Till
>>>
>>> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
>>>
>>>> Thanks Weike and all others for the efforts!
>>>>
>>>> Here comes the latest status, we are in good shape and plan to produce
>>> RC1
>>>> next week.
>>>>
>>>> * Blockers (1 left)
>>>> - [Closed] FLINK-16018 Improve error reporting when submitting batch
>>> job
>>>> (instead of AskTimeoutException)
>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>>> repeated
>>>> job submission
>>>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
>>> when
>>>> use flink-sql-connector-elasticsearch7
>>>> - [Closed] FLINK-16262 Class loader problem with
>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>>>> minimise its OutOfMemoryError
>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
>>> client
>>>> can retrieve JobResult
>>>> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
>>>> throws UnsupportedOperationException
>>>> - [Closed] FLINK-16626 Exception encountered when cancelling a job in
>>>> yarn per-job mode
>>>> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
>> the
>>>> input column is of composite type
>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
>> memory
>>>> state backends
>>>>
>>>> * Critical (1 left)
>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
>>>> with state clean up
>>>> - [Closed] FLINK-16070 Blink planner can not extract correct unique
>> key
>>>> for UpsertStreamTableSink
>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
>>> be
>>>> handled as Fatal Error in TaskManager
>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
>> error
>>>> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
>>> of a
>>>> slot
>>>>
>>>> Please let me know if any concerns/comments. Thanks.
>>>>
>>>> Best Regards,
>>>> Yu
>>>>
>>>>
>>>> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
>>> wrote:
>>>>
>>>>> Hi Yu,
>>>>>
>>>>> Thanks for your updates. I am still working on the fix for
>> FLINK-16626
>>>> and
>>>>> it is expected to be completed by this Sunday after thorough testing.
>>>>>
>>>>> Sincerely,
>>>>> Weike
>>>>>
>>>>> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
>>>>>
>>>>>> Updates for 1.10.1 watched issues (we are in good progress and
>> almost
>>>>>> there
>>>>>> to produce the first RC, thanks all for the efforts):
>>>>>>
>>>>>> * Blockers (3 left)
>>>>>> - [Closed] FLINK-16018 Improve error reporting when submitting
>> batch
>>>> job
>>>>>> (instead of AskTimeoutException)
>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>>>>>> repeated
>>>>>> job submission
>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
>> ClassNotFoundException
>>>> when
>>>>>> use flink-sql-connector-elasticsearch7
>>>>>> - [Closed] FLINK-16262 Class loader problem with
>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>>>>>> minimise its OutOfMemoryError
>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>>>>>> - [PR reviewed] FLINK-16576 State inconsistency on restore with
>>> memory
>>>>>> state backends
>>>>>> - [Under Discussion] FLINK-16626 Exception encountered when
>>>> cancelling a
>>>>>> job in yarn per-job mode
>>>>>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
>>>>>> client
>>>>>> can retrieve JobResult
>>>>>> - [PR reviewed] FLINK-16913
>>>> ReadableConfigToConfigurationAdapter#getEnum
>>>>>> throws UnsupportedOperationException
>>>>>>
>>>>>> * Critical (1 left)
>>>>>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
>>> error
>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>>> results
>>>>>> with state clean up
>>>>>> - [Closed] FLINK-16070 Blink planner can not extract correct
>> unique
>>>> key
>>>>>> for UpsertStreamTableSink
>>>>>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
>>> should
>>>> be
>>>>>> handled as Fatal Error in TaskManager
>>>>>> - [May Postpone] FLINK-16408 Bind user code class loader to
>> lifetime
>>>> of
>>>>>> a
>>>>>> slot
>>>>>>
>>>>>> Please let me know if you find any missing ones, thanks.
>>>>>>
>>>>>> Best Regards,
>>>>>> Yu
>>>>>>
>>>>>>
>>>>>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
>>>>>>
>>>>>>> Here comes the latest status of issues in 1.10.1 watch list:
>>>>>>>
>>>>>>> * Blockers (4 left)
>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
>>>>>> submitting
>>>>>>> batch job (instead of AskTimeoutException)
>>>>>>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>>>>>>> repeated job submission
>>>>>>> - [Closed] FLINK-16170 SearchTemplateRequest
>>> ClassNotFoundException
>>>>>> when
>>>>>>> use flink-sql-connector-elasticsearch7
>>>>>>> - [PR Approved] FLINK-16262 Class loader problem with
>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
>> to
>>>>>>> minimise its OutOfMemoryError
>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>>>>>>> - [In Progress] FLINK-16576 State inconsistency on restore with
>>>> memory
>>>>>>> state backends
>>>>>>> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
>>>>>> MiniCluster
>>>>>>> before client can retrieve JobResult
>>>>>>>
>>>>>>> * Critical (2 left)
>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>>>> results
>>>>>>> with state clean up
>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
>> correct
>>>>>>> unique key for UpsertStreamTableSink
>>>>>>> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
>> should
>>>> be
>>>>>>> handled as Fatal Error in TaskManager
>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime of
>> a
>>>> slot
>>>>>>>
>>>>>>> There's a new blocker added. Please let me know if you find any
>>> other
>>>>>>> issues should be put into the watch list. Thanks.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Yu
>>>>>>>
>>>>>>>
>>>>>>> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> Here is the status update of issues in 1.10.1 watch list:
>>>>>>>>
>>>>>>>> * Blockers (7)
>>>>>>>>
>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting when
>>>>>>>> submitting batch job (instead of AskTimeoutException)
>>>>>>>>
>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
>> OOM
>>>>>> error
>>>>>>>> on repeated job submission
>>>>>>>>
>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>>>>>>>> ClassNotFoundException when use
>> flink-sql-connector-elasticsearch7
>>>>>>>>
>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>>>>>>>
>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
>>> to
>>>>>>>> minimise its OutOfMemoryError
>>>>>>>>
>>>>>>>> - [Closed] FLINK-16454 Update the copyright year in NOTICE
>> files
>>>>>>>>
>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with memory
>>>> state
>>>>>>>> backends
>>>>>>>>
>>>>>>>>
>>>>>>>> * Critical (4)
>>>>>>>>
>>>>>>>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
>>>> results
>>>>>>>> with state clean up
>>>>>>>>
>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not extract
>>> correct
>>>>>>>> unique key for UpsertStreamTableSink
>>>>>>>>
>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
>>> be
>>>>>>>> handled as Fatal Error in TaskManager
>>>>>>>>
>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to lifetime
>> of a
>>>>>> slot
>>>>>>>>
>>>>>>>> Please let me know if you see any new blockers to add to the
>> list.
>>>>>> Thanks.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Yu
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks for the updates Till!
>>>>>>>>>
>>>>>>>>> For FLINK-16018, maybe we could create two sub-tasks for easy
>> and
>>>>>>>>> complete fix separately, and only include the easy one in
>> 1.10.1?
>>> Or
>>>>>> please
>>>>>>>>> just feel free to postpone the whole task to 1.10.2 if "all or
>>>>>> nothing"
>>>>>>>>> policy is preferred (smile). Thanks.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Yu
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
>> trohrmann@apache.org
>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for a soonish bug fix release. Thanks for volunteering as
>> our
>>>>>> release
>>>>>>>>>> manager Yu.
>>>>>>>>>>
>>>>>>>>>> I think we can soon merge the increase of metaspace size and
>>>>>> improving
>>>>>>>>>> the
>>>>>>>>>> error message. The assumption is that we currently don't have
>> too
>>>>>> many
>>>>>>>>>> small Flink 1.10 deployments with a process size <= 1GB. Of
>>> course,
>>>>>> the
>>>>>>>>>> sooner we release the bug fix release, the fewer deployments
>> will
>>>> be
>>>>>>>>>> affected by this change.
>>>>>>>>>>
>>>>>>>>>> For FLINK-16018, I think there would be an easy solution which
>> we
>>>>>> could
>>>>>>>>>> include in the bug fix release. The proper fix will most likely
>>>> take
>>>>>> a
>>>>>>>>>> bit
>>>>>>>>>> longer, though.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Till
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
>>>>>> azagrebin@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
>> user
>>>>>>>>>> mailing
>>>>>>>>>>> list
>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
>>>> Specifically
>>>>>>>>>> asking
>>>>>>>>>>>> for who has very small TM setups?
>>>>>>>>>>>
>>>>>>>>>>> There has been a survey about this topic since 10 days:
>>>>>>>>>>>
>>>>>>>>>>> `[Survey] Default size for the new JVM Metaspace limit in
>> 1.10`
>>>>>>>>>>> I can ask there about the small TM setups.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Another blocker for 1.10.1: FLINK-16576 State inconsistency
>>> on
>>>>>>>>>> restore
>>>>>>>>>>> with
>>>>>>>>>>>> memory state backends
>>>>>>>>>>>>
>>>>>>>>>>>> Let me recompile the watching list with recent feedbacks.
>>>>>> There're
>>>>>>>>>>> totally
>>>>>>>>>>>> 45 issues with Blocker/Critical priority for 1.10.1, out of
>>>>>> which 14
>>>>>>>>>>>> already resolved and 31 left, and the below ones are
>> watched
>>>>>>>>>> (meaning
>>>>>>>>>>> that
>>>>>>>>>>>> once the below ones got fixed, we will kick of the
>> releasing
>>>> with
>>>>>>>>>> left
>>>>>>>>>>> ones
>>>>>>>>>>>> postponed to 1.10.2 unless objections):
>>>>>>>>>>>>
>>>>>>>>>>>> * Blockers (7)
>>>>>>>>>>>> - [Under Discussion] FLINK-16018 Improve error reporting
>>> when
>>>>>>>>>>> submitting
>>>>>>>>>>>> batch job (instead of AskTimeoutException)
>>>>>>>>>>>> - [Under Discussion] FLINK-16142 Memory Leak causes
>>> Metaspace
>>>>>> OOM
>>>>>>>>>> error
>>>>>>>>>>>> on repeated job submission
>>>>>>>>>>>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>>>>>>>>>>> ClassNotFoundException
>>>>>>>>>>>> when use flink-sql-connector-elasticsearch7
>>>>>>>>>>>> - [PR Submitted] FLINK-16262 Class loader problem with
>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
>> directory
>>>>>>>>>>>> - [Closed] FLINK-16406 Increase default value for JVM
>>>>>> Metaspace to
>>>>>>>>>>>> minimise its OutOfMemoryError
>>>>>>>>>>>> - [Open] FLINK-16454 Update the copyright year in NOTICE
>>>> files
>>>>>>>>>>>> - [Open] FLINK-16576 State inconsistency on restore with
>>>> memory
>>>>>>>>>> state
>>>>>>>>>>>> backends
>>>>>>>>>>>>
>>>>>>>>>>>> * Critical (4)
>>>>>>>>>>>> - [Open] FLINK-16047 Blink planner produces wrong
>> aggregate
>>>>>>>>>> results
>>>>>>>>>>> with
>>>>>>>>>>>> state clean up
>>>>>>>>>>>> - [PR Submitted] FLINK-16070 Blink planner can not
>> extract
>>>>>> correct
>>>>>>>>>>> unique
>>>>>>>>>>>> key for UpsertStreamTableSink
>>>>>>>>>>>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
>>>>>> should be
>>>>>>>>>>>> handled as Fatal Error in TaskManager
>>>>>>>>>>>> - [Open] FLINK-16408 Bind user code class loader to
>>> lifetime
>>>>>> of a
>>>>>>>>>> slot
>>>>>>>>>>>>
>>>>>>>>>>>> Please raise your hand if you find any other issues should
>> be
>>>> put
>>>>>>>>>> into
>>>>>>>>>>> this
>>>>>>>>>>>> list, thanks.
>>>>>>>>>>>>
>>>>>>>>>>>> Please also note that 1.10.2 version is already created
>>> (thanks
>>>>>> for
>>>>>>>>>> the
>>>>>>>>>>>> help @jincheng), and feel free to adjust/assign fix version
>>> to
>>>> it
>>>>>>>>>> when
>>>>>>>>>>>> necessary.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Yu
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
>> sewen@apache.org
>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> @Andrey and @Xintong - could we have a quick poll on the
>>> user
>>>>>>>>>> mailing
>>>>>>>>>>>> list
>>>>>>>>>>>>> about increasing the metaspace size in Flink 1.10.1?
>>>>>> Specifically
>>>>>>>>>>> asking
>>>>>>>>>>>>> for who has very small TM setups?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the reminder Stephan and the
>> inputs/discussion
>>>>>>>>>> Andrey and
>>>>>>>>>>>>>> Xintong. I totally agree that we should try best not to
>>>>>>>>>> introduce new
>>>>>>>>>>>>>> issues when resolving the existing one. Will watch
>>> closely
>>>>>>>>>> about the
>>>>>>>>>>>>>> discussion and conclusion around metaspace
>> configuration.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> From the point as a developer instead of RM, I'm
>>> optimistic
>>>>>> that
>>>>>>>>>>>>> reserving
>>>>>>>>>>>>>> 160MB more memory to JVM metaspace by default won't
>>> cause a
>>>>>> big
>>>>>>>>>>>> problem,
>>>>>>>>>>>>>> based on the assumption that nowadays assigning 2GB/4GB
>>>>>> memory
>>>>>>>>>> per TM
>>>>>>>>>>>> is
>>>>>>>>>>>>>> the common case.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, 13 Mar 2020 at 09:54, Xintong Song <
>>>>>>>>>> tonysong820@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think in FLINK-16406 we have not only increased the
>>>>>> default
>>>>>>>>>> value
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> metaspace, but also increased the 'process.size' in
>>>> default
>>>>>>>>>>>>>>> 'flink-conf.yaml'.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> That means:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - For people using the default 'flink-conf.yaml',
>>> it's
>>>>>> not
>>>>>>>>>>> really
>>>>>>>>>>>>>> taking
>>>>>>>>>>>>>>> memory from heap / managed for metaspace.
>>>>>>>>>>>>>>> - For people using custom configuration files:
>>>>>>>>>>>>>>> - If they are specifying 'flink.size', then
>>>> metaspace
>>>>>>>>>> does
>>>>>>>>>>> not
>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>> affect the flink memory internal components
>>> (heap,
>>>>>>>>>> managed,
>>>>>>>>>>>>> etc.).
>>>>>>>>>>>>>>> - If they are specifying 'process.size' (most
>>>> likely
>>>>>>>>>> active
>>>>>>>>>>>>>> setups),
>>>>>>>>>>>>>>> then they will have less heap / managed memory,
>>> or
>>>>>>>>>> worse,
>>>>>>>>>>> their
>>>>>>>>>>>>>>> configuration may not work without adjustment.
>>>>>>>>>>>>>>> - For a TM with large process memory (e.g.,
>> 4
>>> GB
>>>>>> or
>>>>>>>>>> more),
>>>>>>>>>>>> it
>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> probably ok that the metaspace takes 160 MB
>>> more
>>>>>>>>>> memory.
>>>>>>>>>>>>>>> - For a TM with smaller total memory (1 or 2
>>> GB)
>>>>>> the
>>>>>>>>>>>> metaspace
>>>>>>>>>>>>>>> increase is more likely to cause problem.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So basically only people have small 'process.size' in
>>>>>> custom
>>>>>>>>>> config
>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> are really affected. I'm not sure what is the
>>> proportion
>>>> of
>>>>>>>>>> such
>>>>>>>>>>> use
>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>> though. (From questions asked on the user ML,
>> probably
>>>> not
>>>>>>>>>> much).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank you~
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Xintong Song
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
>>>>>>>>>> sewen@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No need to revert it now - I am not saying it
>> should
>>>> not
>>>>>> go
>>>>>>>>>> into
>>>>>>>>>>>>>> 1.10.1,
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> am just saying this is not clear to me yet.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We have to trade off the fact that we may break
>> some
>>>>>>>>>> deployments
>>>>>>>>>>>> with
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> fact that we will "safe" a lot of new deployments.
>>>>>>>>>>>>>>>> I simply lack the data points / insight at the
>> moment
>>>> to
>>>>>>>>>>> understand
>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>> significant both cases are, meaning how many users
>>>> would
>>>>>> be
>>>>>>>>>>>> affected
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> how badly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Independent of that, improving the error message is
>>>>>> always
>>>>>>>>>>> helpful.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
>> because
>>>>>> users
>>>>>>>>>> expect
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>>> that without config adjustments. That might not
>>> be
>>>>>>>>>> possible
>>>>>>>>>>>> with
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> change.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok, makes sense, I will revert it for 1.10 and
>> only
>>>>>> try to
>>>>>>>>>>>> improve
>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>> message and docs.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 13:15, Stephan Ewen <
>>>>>>>>>> sewen@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> @Andrey about the increase in metaspace size
>>>>>>>>>>>>>>>>>> - I have no concerns for 1.11.0.
>>>>>>>>>>>>>>>>>> - For 1.10.1 I am not completely sure,
>> because
>>>>>> users
>>>>>>>>>> expect
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>>> that without config adjustments. That might not
>>> be
>>>>>>>>>> possible
>>>>>>>>>>>> with
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> change.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 12:55 PM Andrey
>> Zagrebin
>>> <
>>>>>>>>>>>>>>>>> azagrebin.apache@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
>> Metaspace
>>>> OOM
>>>>>>>>>> error
>>>>>>>>>>> on
>>>>>>>>>>>>>>> repeated
>>>>>>>>>>>>>>>>>>> job”
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> My understanding that the issue is basically
>>>> covered
>>>>>>>>>> by:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
>>> be
>>>>>>>>>> handled as
>>>>>>>>>>>>> Fatal
>>>>>>>>>>>>>>>> Error
>>>>>>>>>>>>>>>>>>> in TaskManager
>>>>>>>>>>>>>>>>>>> no full consensus there but improving error
>>>>>> message
>>>>>>>>>> for
>>>>>>>>>>>>> existing
>>>>>>>>>>>>>>>> task
>>>>>>>>>>>>>>>>>>> thread fatal handling could be done at least
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - [FLINK-16406] Increase default value for JVM
>>>>>>>>>> Metaspace to
>>>>>>>>>>>>>> minimise
>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>> OutOfMemoryError
>>>>>>>>>>>>>>>>>>> see further
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - [FLINK-16246] Exclude
>>> "SdkMBeanRegistrySupport"
>>>>>> from
>>>>>>>>>>>>> dynamically
>>>>>>>>>>>>>>>>> loaded
>>>>>>>>>>>>>>>>>>> AWS connectors
>>>>>>>>>>>>>>>>>>> not sure whether this is a blocker but looks
>>>> close
>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>>> resolved
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
>>> JVM
>>>>>>>>>>> Metaspace"
>>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
>>>> bugfix
>>>>>>>>>> release?
>>>>>>>>>>>> It
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
>>>>>> memory
>>>>>>>>>> on
>>>>>>>>>>>>> existing
>>>>>>>>>>>>>>>> setups
>>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> My understanding was that increasing to 256m
>>>>>> resolved
>>>>>>>>>> the
>>>>>>>>>>>>> reported
>>>>>>>>>>>>>>>>> problems
>>>>>>>>>>>>>>>>>>> and we decided to make the change so I have
>>> merged
>>>>>> it
>>>>>>>>>> today
>>>>>>>>>>> as
>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>> were
>>>>>>>>>>>>>>>>>>> no more concerns.
>>>>>>>>>>>>>>>>>>> If there are concerns I can revert it.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On the other hand, I think improving the
>> message
>>>>>> error
>>>>>>>>>> with
>>>>>>>>>>>>>>> reference
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the metaspace option should help the most
>>>>>>>>>>>>>>>>>>> because user would not have to read all docs
>> to
>>>> fix
>>>>>> it
>>>>>>>>>>>>>>>>>>> then maybe this change is not even needed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>> Andrey
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>>>>>>>>>> sewen@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Good idea to go ahead with 1.10.1
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> About "FLINK-16142 Memory Leak causes
>> Metaspace
>>>> OOM
>>>>>>>>>> error
>>>>>>>>>>> on
>>>>>>>>>>>>>>> repeated
>>>>>>>>>>>>>>>>>>> job"
>>>>>>>>>>>>>>>>>>>> - I don't think we have consensus on the
>> exact
>>>>>>>>>> solution,
>>>>>>>>>>> yet,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> the changes might also have side effects that
>>> are
>>>>>>>>>> hard to
>>>>>>>>>>>>>> predict,
>>>>>>>>>>>>>>>> so I
>>>>>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>>>>>> not sure we should rush this in.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> About "FLINK-16406 Increase default value for
>>> JVM
>>>>>>>>>>> Metaspace"
>>>>>>>>>>>>>>>>>>>> - Have we consensus that this is okay for a
>>>> bugfix
>>>>>>>>>> release?
>>>>>>>>>>>> It
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>>>> setups, takes away memory from heap / managed
>>>>>> memory
>>>>>>>>>> on
>>>>>>>>>>>>> existing
>>>>>>>>>>>>>>>> setups
>>>>>>>>>>>>>>>>>>>> that keep their flink-conf.yaml.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> We may need to unblock the release form these
>>> two
>>>>>>>>>> issues
>>>>>>>>>>> and
>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>> having 1.10.2 in the near future.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
>>>>>>>>>> carp84@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks for the reminder Jark. Will keep an
>> eye
>>>> on
>>>>>>>>>> these
>>>>>>>>>>> two.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>>>>>>>>>> imjark@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
>>>>>>>>>>>>>>>>>>>>>> +1 to start 1.10.1 release cycle.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> From the Table SQL module, I think we
>> should
>>>> also
>>>>>>>>>> try to
>>>>>>>>>>>> get
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> following issues:
>>>>>>>>>>>>>>>>>>>>>> - FLINK-16441: Allow users to override
>>>> flink-conf
>>>>>>>>>>>> parameters
>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> SQL
>>>>>>>>>>>>>>>>>>> CLI
>>>>>>>>>>>>>>>>>>>>>> environment
>>>>>>>>>>>>>>>>>>>>>> this allows users to set e.g. statebackend,
>>>>>>>>>> watermark
>>>>>>>>>>>>> interval,
>>>>>>>>>>>>>>>>>>>>>> exactly-once/at-least-once, in the SQL CLI
>>>>>>>>>>>>>>>>>>>>>> - FLINK-16217: SQL Client crashed when any
>>>>>> uncatched
>>>>>>>>>>>>> exception
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> thrown
>>>>>>>>>>>>>>>>>>>>>> this will improve much experience when
>> using
>>>> SQL
>>>>>> CLI
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>> Jark
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
>>>>>>>>>> carp84@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thanks for the suggestion Andrey! I've
>> added
>>>>>>>>>> 1.10.1 into
>>>>>>>>>>>>>>>> FLINK-16225
>>>>>>>>>>>>>>>>>>>>> fix
>>>>>>>>>>>>>>>>>>>>>>> versions and promoted its priority to
>>>> Critical.
>>>>>>>>>> Will
>>>>>>>>>>> also
>>>>>>>>>>>>>> watch
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> progress of FLINK-16108/FLINK-16408.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
>>> Zagrebin
>>>> <
>>>>>>>>>>>>>>>> azagrebin@apache.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks for kicking off the 1.10.1 release
>>>>>>>>>> discussion!
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Apart from
>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
>>> JVM
>>>>>>>>>> Metaspace
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> minimise
>>>>>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
>>>>>>>>>>>>>>>>>>>>>>>> which should be merged soon
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I think we should also try to get in the
>>>>>> following
>>>>>>>>>>>> issues:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16225] Metaspace Out Of Memory
>>>> should
>>>>>> be
>>>>>>>>>>> handled
>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> Fatal
>>>>>>>>>>>>>>>>>>>>>> Error
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>> TaskManager
>>>>>>>>>>>>>>>>>>>>>>>> This should solve the Metaspace problem
>>> even
>>>>>> in a
>>>>>>>>>>> better
>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>>>>>> OOM
>>>>>>>>>>>>>>>>>>>>>>>> failure should point users to the docs
>>>>>> immediately
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16408] Bind user code class
>> loader
>>>> to
>>>>>>>>>> lifetime
>>>>>>>>>>>> of
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> slot
>>>>>>>>>>>>>>>>>>>>>>>> This should give a better protection
>>> against
>>>>>> class
>>>>>>>>>>>> loading
>>>>>>>>>>>>>>> leaks
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> - [FLINK-16018] Improve error reporting
>>> when
>>>>>>>>>> submitting
>>>>>>>>>>>>> batch
>>>>>>>>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>>> (instead
>>>>>>>>>>>>>>>>>>>>>>>> of AskTimeoutException)
>>>>>>>>>>>>>>>>>>>>>>>> This problem has recently happened for
>>>> multiple
>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>> Andrey
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
>>> Li <
>>>>>>>>>>>>>>>>> jingsonglee0@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving. Yu. +1 for starting
>>> the
>>>>>>>>>> 1.10.1
>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Some issues are very important, Users
>> are
>>>>>> looking
>>>>>>>>>>>> forward
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>> Jingsong Lee
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
>>> Guo <
>>>>>>>>>>>>>>> karmagyz@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release, Yu!
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for starting the 1.10.1 release
>> cycle.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>>> Yangze Guo
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
>>>> Song
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the explanation.
>>>>>>>>>>>>>>>>>>>>>>>>>>> I've no concerns. I was just trying to
>>> get
>>>>>> some
>>>>>>>>>>> inputs
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>> prioritizing
>>>>>>>>>>>>>>>>>>>>>>>>>>> tasks on my side, and ~1month sounds
>>> good
>>>> to
>>>>>>>>>> me.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
>> Li <
>>>>>>>>>>>>> carp84@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> bq. what is the time plan for 1.10.1?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> According to the history, the first
>>> patch
>>>>>>>>>> release
>>>>>>>>>>> of
>>>>>>>>>>>> a
>>>>>>>>>>>>>>> major
>>>>>>>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>>>>>>>>> take ~1month from discussion started,
>>>>>>>>>> depending on
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> speed
>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>> blocker
>>>>>>>>>>>>>>>>>>>>>>>>>>>> issue resolving:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.8.1: started discussion on May
>> 28th
>>>>>> [1],
>>>>>>>>>>> released
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> Jul
>>>>>>>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>>>>>>>>> * 1.9.1: started discussion on Sep
>> 23rd
>>>>>> [3],
>>>>>>>>>>> released
>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> Oct
>>>>>>>>>>>>>>>>>>>>>> 19th
>>>>>>>>>>>>>>>>>>>>>>>> [4]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> We won't rush to match the history of
>>>>>> course,
>>>>>>>>>> but
>>>>>>>>>>>> could
>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>> as a
>>>>>>>>>>>>>>>>>>>>>>>>>>>> reference. And please feel free to
>> let
>>> me
>>>>>>>>>> know if
>>>>>>>>>>> any
>>>>>>>>>>>>>>>>>>>>> concerns
>>>>>>>>>>>>>>>>>>>>>>>>> Xintong.
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [4]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
>>>> Song
>>>>>> <
>>>>>>>>>>>>>>>>>>>>>>> tonysong820@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Yu, for the kick off and
>>>>>> volunteering
>>>>>>>>>> to be
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>> manager.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 for the proposal.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> One quick question, what is the time
>>>> plan
>>>>>> for
>>>>>>>>>>>> 1.10.1?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you~
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xintong Song
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
>>>> Zhijiang
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <wangzhijiang999@aliyun.com
>> .invalid>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for driving this release,
>> Yu!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +1 on my side
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Zhijiang
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>> ------------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
>> Flink
>>>>>> 1.10.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the supplement Hequn.
>> Yes
>>>> will
>>>>>>>>>> also
>>>>>>>>>>> keep
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>> eye
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> existing blocker issues.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
>>>>>> Cheng <
>>>>>>>>>>>>>>>>>>>>>> hequn@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Yu,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks a lot for raising the
>>>> discussion
>>>>>> and
>>>>>>>>>>>>> volunteer
>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> manager!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I found there are some other
>>> issues[1]
>>>>>>>>>> which are
>>>>>>>>>>>>>> marked
>>>>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>>>>> blocker:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16454 Update the copyright
>>>> year
>>>>>> in
>>>>>>>>>>> NOTICE
>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16262 Class loader problem
>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
>>>>>>>>>> and
>>>>>>>>>>>> usrlib
>>>>>>>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16170
>> SearchTemplateRequest
>>>>>>>>>>>>>>>>>>>>>> ClassNotFoundException
>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16018 Improve error
>>> reporting
>>>>>> when
>>>>>>>>>>>>> submitting
>>>>>>>>>>>>>>>>>>>>>> batch
>>>>>>>>>>>>>>>>>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (instead
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AskTimeoutException)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> These may also need to be resolved
>>> in
>>>>>>>>>> 1.10.1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hequn
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>
>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
>>> Li
>>>> <
>>>>>>>>>>>>>>>>>>>>> carp84@gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Jincheng,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, your help would be very
>>> helpful.
>>>>>>>>>> Thanks a
>>>>>>>>>>>> lot!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
>>>> jincheng
>>>>>>>>>> sun <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> sunjincheng121@gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for bring up the
>> discussion
>>>>>> Yu. I
>>>>>>>>>> would
>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>> give
>>>>>>>>>>>>>>>>>>>>>>>>>> you a
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hand
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the last stage when the RC is
>>>>>>>>>> finished.(If you
>>>>>>>>>>>>>>>>>>>>> need)
>>>>>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jincheng
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
>>>>>> 于2020年3月10日周二
>>>>>>>>>>>> 下午5:49写道：
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It has been almost one month
>>> since
>>>> we
>>>>>>>>>>> released
>>>>>>>>>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.0. We
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have more than 40 resolved
>>>>>>>>>> improvements/bugs
>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.10
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and I propose to start the
>> 1.10.1
>>>>>>>>>> release
>>>>>>>>>>>> cycle.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Most noticeable fixes are:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove
>> the
>>>>>>>>>> license and
>>>>>>>>>>>>>>>>>>>>> notice
>>>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-ml-lib
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> module
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB
>>> resource
>>>>>> leak
>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flink-state-processor-api
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
>>>> should
>>>>>> be
>>>>>>>>>> known
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> HiveCatalog
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-2336
>>>>>>>>>> ArrayIndexOufOBoundsException in
>>>>>>>>>>>>>>>>>>>>>>>>> TypeExtractor
>>>>>>>>>>>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapping
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample
>> is
>>>>>>>>>> failed if
>>>>>>>>>>>>>>>>>>>>> running
>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>> blink
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> planner
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
>>>> constraints
>>>>>>>>>> are not
>>>>>>>>>>>>>>>>>>>>> reset
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>> task
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recovery
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> DefaultScheduler
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
>>>>>> function
>>>>>>>>>> using
>>>>>>>>>>>> sql
>>>>>>>>>>>>>>>>>>>>>>>> casuing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ValidationException: SQL
>>> validation
>>>>>>>>>> failed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Furthermore, I think the
>>> following
>>>>>>>>>> issues
>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>>>> merged
>>>>>>>>>>>>>>>>>>>>>>>>>>>> before
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> release (especially the
>> Metaspace
>>>> OOM
>>>>>>>>>> issue):
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak
>> causes
>>>>>>>>>> Metaspace
>>>>>>>>>>> OOM
>>>>>>>>>>>>>>>>>>>>>> error
>>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>>>>>>>> repeated
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> job
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> submission
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
>>>> value
>>>>>>>>>> for JVM
>>>>>>>>>>>>>>>>>>>>>>> Metaspace
>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> minimise
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> OutOfMemoryError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
>>>> produces
>>>>>>>>>> wrong
>>>>>>>>>>>>>>>>>>>>>> aggregate
>>>>>>>>>>>>>>>>>>>>>>>>>> results
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> state
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clean up
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can
>>> not
>>>>>>>>>> extract
>>>>>>>>>>>>>>>>>>>>> correct
>>>>>>>>>>>>>>>>>>>>>>>>> unique
>>>>>>>>>>>>>>>>>>>>>>>>>> key
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> UpsertStreamTableSink
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I would volunteer as the
>> release
>>>>>>>>>> manager and
>>>>>>>>>>>> kick
>>>>>>>>>>>>>>>>>>>>>> off
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> process
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> once blocker issues are merged.
>>>> What
>>>>>> do
>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>> think?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If there are any concerns or
>>>> missing
>>>>>>>>>> blocker
>>>>>>>>>>>>>>>>>>>>> issues
>>>>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>>>>>>> to be
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1.10.1, please let me know.
>>> Thanks.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> Best, Jingsong Lee
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by jincheng sun <su...@gmail.com>.
Looking forward the first RC of Flink 1.10.1 .
Good job Yu!
Best,
Jincheng
Jark Wu <im...@gmail.com> 于2020年4月15日周三 下午6:28写道：
> +1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
> released.
>
> Best,
> Jark
>
> On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org> wrote:
>
> > Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
> > lot for driving this effort Yu!
> >
> > Cheers,
> > Till
> >
> > On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> >
> > > Thanks Weike and all others for the efforts!
> > >
> > > Here comes the latest status, we are in good shape and plan to produce
> > RC1
> > > next week.
> > >
> > > * Blockers (1 left)
> > > - [Closed] FLINK-16018 Improve error reporting when submitting batch
> > job
> > > (instead of AskTimeoutException)
> > > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > repeated
> > > job submission
> > > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> > when
> > > use flink-sql-connector-elasticsearch7
> > > - [Closed] FLINK-16262 Class loader problem with
> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > > minimise its OutOfMemoryError
> > > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > > - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> > client
> > > can retrieve JobResult
> > > - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> > > throws UnsupportedOperationException
> > > - [Closed] FLINK-16626 Exception encountered when cancelling a job in
> > > yarn per-job mode
> > > - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when
> the
> > > input column is of composite type
> > > - [PR reviewed] FLINK-16576 State inconsistency on restore with
> memory
> > > state backends
> > >
> > > * Critical (1 left)
> > > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> > > with state clean up
> > > - [Closed] FLINK-16070 Blink planner can not extract correct unique
> key
> > > for UpsertStreamTableSink
> > > - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
> > be
> > > handled as Fatal Error in TaskManager
> > > - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> error
> > > - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> > of a
> > > slot
> > >
> > > Please let me know if any concerns/comments. Thanks.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> > wrote:
> > >
> > > > Hi Yu,
> > > >
> > > > Thanks for your updates. I am still working on the fix for
> FLINK-16626
> > > and
> > > > it is expected to be completed by this Sunday after thorough testing.
> > > >
> > > > Sincerely,
> > > > Weike
> > > >
> > > > On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> > > >
> > > >> Updates for 1.10.1 watched issues (we are in good progress and
> almost
> > > >> there
> > > >> to produce the first RC, thanks all for the efforts):
> > > >>
> > > >> * Blockers (3 left)
> > > >> - [Closed] FLINK-16018 Improve error reporting when submitting
> batch
> > > job
> > > >> (instead of AskTimeoutException)
> > > >> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > >> repeated
> > > >> job submission
> > > >> - [Closed] FLINK-16170 SearchTemplateRequest
> ClassNotFoundException
> > > when
> > > >> use flink-sql-connector-elasticsearch7
> > > >> - [Closed] FLINK-16262 Class loader problem with
> > > >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > > >> minimise its OutOfMemoryError
> > > >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > > >> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> > memory
> > > >> state backends
> > > >> - [Under Discussion] FLINK-16626 Exception encountered when
> > > cancelling a
> > > >> job in yarn per-job mode
> > > >> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> > > >> client
> > > >> can retrieve JobResult
> > > >> - [PR reviewed] FLINK-16913
> > > ReadableConfigToConfigurationAdapter#getEnum
> > > >> throws UnsupportedOperationException
> > > >>
> > > >> * Critical (1 left)
> > > >> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> > error
> > > >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > results
> > > >> with state clean up
> > > >> - [Closed] FLINK-16070 Blink planner can not extract correct
> unique
> > > key
> > > >> for UpsertStreamTableSink
> > > >> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> > should
> > > be
> > > >> handled as Fatal Error in TaskManager
> > > >> - [May Postpone] FLINK-16408 Bind user code class loader to
> lifetime
> > > of
> > > >> a
> > > >> slot
> > > >>
> > > >> Please let me know if you find any missing ones, thanks.
> > > >>
> > > >> Best Regards,
> > > >> Yu
> > > >>
> > > >>
> > > >> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> > > >>
> > > >> > Here comes the latest status of issues in 1.10.1 watch list:
> > > >> >
> > > >> > * Blockers (4 left)
> > > >> > - [Under Discussion] FLINK-16018 Improve error reporting when
> > > >> submitting
> > > >> > batch job (instead of AskTimeoutException)
> > > >> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > >> > repeated job submission
> > > >> > - [Closed] FLINK-16170 SearchTemplateRequest
> > ClassNotFoundException
> > > >> when
> > > >> > use flink-sql-connector-elasticsearch7
> > > >> > - [PR Approved] FLINK-16262 Class loader problem with
> > > >> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > >> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> to
> > > >> > minimise its OutOfMemoryError
> > > >> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > > >> > - [In Progress] FLINK-16576 State inconsistency on restore with
> > > memory
> > > >> > state backends
> > > >> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> > > >> MiniCluster
> > > >> > before client can retrieve JobResult
> > > >> >
> > > >> > * Critical (2 left)
> > > >> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > > results
> > > >> > with state clean up
> > > >> > - [PR Submitted] FLINK-16070 Blink planner can not extract
> correct
> > > >> > unique key for UpsertStreamTableSink
> > > >> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory
> should
> > > be
> > > >> > handled as Fatal Error in TaskManager
> > > >> > - [Open] FLINK-16408 Bind user code class loader to lifetime of
> a
> > > slot
> > > >> >
> > > >> > There's a new blocker added. Please let me know if you find any
> > other
> > > >> > issues should be put into the watch list. Thanks.
> > > >> >
> > > >> > Best Regards,
> > > >> > Yu
> > > >> >
> > > >> >
> > > >> > On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> > > >> >
> > > >> >> Hi All,
> > > >> >>
> > > >> >> Here is the status update of issues in 1.10.1 watch list:
> > > >> >>
> > > >> >> * Blockers (7)
> > > >> >>
> > > >> >> - [Under Discussion] FLINK-16018 Improve error reporting when
> > > >> >> submitting batch job (instead of AskTimeoutException)
> > > >> >>
> > > >> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> OOM
> > > >> error
> > > >> >> on repeated job submission
> > > >> >>
> > > >> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > > >> >> ClassNotFoundException when use
> flink-sql-connector-elasticsearch7
> > > >> >>
> > > >> >> - [PR Submitted] FLINK-16262 Class loader problem with
> > > >> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > >> >>
> > > >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> > to
> > > >> >> minimise its OutOfMemoryError
> > > >> >>
> > > >> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE
> files
> > > >> >>
> > > >> >> - [Open] FLINK-16576 State inconsistency on restore with memory
> > > state
> > > >> >> backends
> > > >> >>
> > > >> >>
> > > >> >> * Critical (4)
> > > >> >>
> > > >> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > > results
> > > >> >> with state clean up
> > > >> >>
> > > >> >> - [PR Submitted] FLINK-16070 Blink planner can not extract
> > correct
> > > >> >> unique key for UpsertStreamTableSink
> > > >> >>
> > > >> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> > be
> > > >> >> handled as Fatal Error in TaskManager
> > > >> >>
> > > >> >> - [Open] FLINK-16408 Bind user code class loader to lifetime
> of a
> > > >> slot
> > > >> >>
> > > >> >> Please let me know if you see any new blockers to add to the
> list.
> > > >> Thanks.
> > > >> >>
> > > >> >> Best Regards,
> > > >> >> Yu
> > > >> >>
> > > >> >>
> > > >> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> > > >> >>
> > > >> >>> Thanks for the updates Till!
> > > >> >>>
> > > >> >>> For FLINK-16018, maybe we could create two sub-tasks for easy
> and
> > > >> >>> complete fix separately, and only include the easy one in
> 1.10.1?
> > Or
> > > >> please
> > > >> >>> just feel free to postpone the whole task to 1.10.2 if "all or
> > > >> nothing"
> > > >> >>> policy is preferred (smile). Thanks.
> > > >> >>>
> > > >> >>> Best Regards,
> > > >> >>> Yu
> > > >> >>>
> > > >> >>>
> > > >> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <
> trohrmann@apache.org
> > >
> > > >> >>> wrote:
> > > >> >>>
> > > >> >>>> +1 for a soonish bug fix release. Thanks for volunteering as
> our
> > > >> release
> > > >> >>>> manager Yu.
> > > >> >>>>
> > > >> >>>> I think we can soon merge the increase of metaspace size and
> > > >> improving
> > > >> >>>> the
> > > >> >>>> error message. The assumption is that we currently don't have
> too
> > > >> many
> > > >> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> > course,
> > > >> the
> > > >> >>>> sooner we release the bug fix release, the fewer deployments
> will
> > > be
> > > >> >>>> affected by this change.
> > > >> >>>>
> > > >> >>>> For FLINK-16018, I think there would be an easy solution which
> we
> > > >> could
> > > >> >>>> include in the bug fix release. The proper fix will most likely
> > > take
> > > >> a
> > > >> >>>> bit
> > > >> >>>> longer, though.
> > > >> >>>>
> > > >> >>>> Cheers,
> > > >> >>>> Till
> > > >> >>>>
> > > >> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> > > >> azagrebin@apache.org>
> > > >> >>>> wrote:
> > > >> >>>>
> > > >> >>>> > > @Andrey and @Xintong - could we have a quick poll on the
> user
> > > >> >>>> mailing
> > > >> >>>> > list
> > > >> >>>> > > about increasing the metaspace size in Flink 1.10.1?
> > > Specifically
> > > >> >>>> asking
> > > >> >>>> > > for who has very small TM setups?
> > > >> >>>> >
> > > >> >>>> > There has been a survey about this topic since 10 days:
> > > >> >>>> >
> > > >> >>>> > `[Survey] Default size for the new JVM Metaspace limit in
> 1.10`
> > > >> >>>> > I can ask there about the small TM setups.
> > > >> >>>> >
> > > >> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> > wrote:
> > > >> >>>> >
> > > >> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency
> > on
> > > >> >>>> restore
> > > >> >>>> > with
> > > >> >>>> > > memory state backends
> > > >> >>>> > >
> > > >> >>>> > > Let me recompile the watching list with recent feedbacks.
> > > >> There're
> > > >> >>>> > totally
> > > >> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of
> > > >> which 14
> > > >> >>>> > > already resolved and 31 left, and the below ones are
> watched
> > > >> >>>> (meaning
> > > >> >>>> > that
> > > >> >>>> > > once the below ones got fixed, we will kick of the
> releasing
> > > with
> > > >> >>>> left
> > > >> >>>> > ones
> > > >> >>>> > > postponed to 1.10.2 unless objections):
> > > >> >>>> > >
> > > >> >>>> > > * Blockers (7)
> > > >> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting
> > when
> > > >> >>>> > submitting
> > > >> >>>> > > batch job (instead of AskTimeoutException)
> > > >> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes
> > Metaspace
> > > >> OOM
> > > >> >>>> error
> > > >> >>>> > > on repeated job submission
> > > >> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > > >> >>>> > ClassNotFoundException
> > > >> >>>> > > when use flink-sql-connector-elasticsearch7
> > > >> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
> > > >> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> directory
> > > >> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM
> > > >> Metaspace to
> > > >> >>>> > > minimise its OutOfMemoryError
> > > >> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE
> > > files
> > > >> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with
> > > memory
> > > >> >>>> state
> > > >> >>>> > > backends
> > > >> >>>> > >
> > > >> >>>> > > * Critical (4)
> > > >> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong
> aggregate
> > > >> >>>> results
> > > >> >>>> > with
> > > >> >>>> > > state clean up
> > > >> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not
> extract
> > > >> correct
> > > >> >>>> > unique
> > > >> >>>> > > key for UpsertStreamTableSink
> > > >> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> > > >> should be
> > > >> >>>> > > handled as Fatal Error in TaskManager
> > > >> >>>> > > - [Open] FLINK-16408 Bind user code class loader to
> > lifetime
> > > >> of a
> > > >> >>>> slot
> > > >> >>>> > >
> > > >> >>>> > > Please raise your hand if you find any other issues should
> be
> > > put
> > > >> >>>> into
> > > >> >>>> > this
> > > >> >>>> > > list, thanks.
> > > >> >>>> > >
> > > >> >>>> > > Please also note that 1.10.2 version is already created
> > (thanks
> > > >> for
> > > >> >>>> the
> > > >> >>>> > > help @jincheng), and feel free to adjust/assign fix version
> > to
> > > it
> > > >> >>>> when
> > > >> >>>> > > necessary.
> > > >> >>>> > >
> > > >> >>>> > > Best Regards,
> > > >> >>>> > > Yu
> > > >> >>>> > >
> > > >> >>>> > >
> > > >> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <
> sewen@apache.org
> > >
> > > >> >>>> wrote:
> > > >> >>>> > >
> > > >> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the
> > user
> > > >> >>>> mailing
> > > >> >>>> > > list
> > > >> >>>> > > > about increasing the metaspace size in Flink 1.10.1?
> > > >> Specifically
> > > >> >>>> > asking
> > > >> >>>> > > > for who has very small TM setups?
> > > >> >>>> > > >
> > > >> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> > > >> wrote:
> > > >> >>>> > > >
> > > >> >>>> > > > > Thanks for the reminder Stephan and the
> inputs/discussion
> > > >> >>>> Andrey and
> > > >> >>>> > > > > Xintong. I totally agree that we should try best not to
> > > >> >>>> introduce new
> > > >> >>>> > > > > issues when resolving the existing one. Will watch
> > closely
> > > >> >>>> about the
> > > >> >>>> > > > > discussion and conclusion around metaspace
> configuration.
> > > >> >>>> > > > >
> > > >> >>>> > > > > From the point as a developer instead of RM, I'm
> > optimistic
> > > >> that
> > > >> >>>> > > > reserving
> > > >> >>>> > > > > 160MB more memory to JVM metaspace by default won't
> > cause a
> > > >> big
> > > >> >>>> > > problem,
> > > >> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB
> > > >> memory
> > > >> >>>> per TM
> > > >> >>>> > > is
> > > >> >>>> > > > > the common case.
> > > >> >>>> > > > >
> > > >> >>>> > > > > Best Regards,
> > > >> >>>> > > > > Yu
> > > >> >>>> > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> > > >> >>>> tonysong820@gmail.com>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > >
> > > >> >>>> > > > > > I think in FLINK-16406 we have not only increased the
> > > >> default
> > > >> >>>> value
> > > >> >>>> > > for
> > > >> >>>> > > > > > metaspace, but also increased the 'process.size' in
> > > default
> > > >> >>>> > > > > > 'flink-conf.yaml'.
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > That means:
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > - For people using the default 'flink-conf.yaml',
> > it's
> > > >> not
> > > >> >>>> > really
> > > >> >>>> > > > > taking
> > > >> >>>> > > > > > memory from heap / managed for metaspace.
> > > >> >>>> > > > > > - For people using custom configuration files:
> > > >> >>>> > > > > > - If they are specifying 'flink.size', then
> > > metaspace
> > > >> >>>> does
> > > >> >>>> > not
> > > >> >>>> > > > > really
> > > >> >>>> > > > > > affect the flink memory internal components
> > (heap,
> > > >> >>>> managed,
> > > >> >>>> > > > etc.).
> > > >> >>>> > > > > > - If they are specifying 'process.size' (most
> > > likely
> > > >> >>>> active
> > > >> >>>> > > > > setups),
> > > >> >>>> > > > > > then they will have less heap / managed memory,
> > or
> > > >> >>>> worse,
> > > >> >>>> > their
> > > >> >>>> > > > > > configuration may not work without adjustment.
> > > >> >>>> > > > > > - For a TM with large process memory (e.g.,
> 4
> > GB
> > > >> or
> > > >> >>>> more),
> > > >> >>>> > > it
> > > >> >>>> > > > is
> > > >> >>>> > > > > > probably ok that the metaspace takes 160 MB
> > more
> > > >> >>>> memory.
> > > >> >>>> > > > > > - For a TM with smaller total memory (1 or 2
> > GB)
> > > >> the
> > > >> >>>> > > metaspace
> > > >> >>>> > > > > > increase is more likely to cause problem.
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > So basically only people have small 'process.size' in
> > > >> custom
> > > >> >>>> config
> > > >> >>>> > > > file
> > > >> >>>> > > > > > are really affected. I'm not sure what is the
> > proportion
> > > of
> > > >> >>>> such
> > > >> >>>> > use
> > > >> >>>> > > > > cases
> > > >> >>>> > > > > > though. (From questions asked on the user ML,
> probably
> > > not
> > > >> >>>> much).
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > Thank you~
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > Xintong Song
> > > >> >>>> > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> > > >> >>>> sewen@apache.org>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > > >
> > > >> >>>> > > > > > > No need to revert it now - I am not saying it
> should
> > > not
> > > >> go
> > > >> >>>> into
> > > >> >>>> > > > > 1.10.1,
> > > >> >>>> > > > > > I
> > > >> >>>> > > > > > > am just saying this is not clear to me yet.
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > > > We have to trade off the fact that we may break
> some
> > > >> >>>> deployments
> > > >> >>>> > > with
> > > >> >>>> > > > > the
> > > >> >>>> > > > > > > fact that we will "safe" a lot of new deployments.
> > > >> >>>> > > > > > > I simply lack the data points / insight at the
> moment
> > > to
> > > >> >>>> > understand
> > > >> >>>> > > > how
> > > >> >>>> > > > > > > significant both cases are, meaning how many users
> > > would
> > > >> be
> > > >> >>>> > > affected
> > > >> >>>> > > > > and
> > > >> >>>> > > > > > > how badly.
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > > > Independent of that, improving the error message is
> > > >> always
> > > >> >>>> > helpful.
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > >> >>>> > > > > > > azagrebin.apache@gmail.com>
> > > >> >>>> > > > > > > wrote:
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure,
> because
> > > >> users
> > > >> >>>> expect
> > > >> >>>> > > to
> > > >> >>>> > > > > > > upgrade
> > > >> >>>> > > > > > > > > that without config adjustments. That might not
> > be
> > > >> >>>> possible
> > > >> >>>> > > with
> > > >> >>>> > > > > that
> > > >> >>>> > > > > > > > > change.
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and
> only
> > > >> try to
> > > >> >>>> > > improve
> > > >> >>>> > > > > > error
> > > >> >>>> > > > > > > > message and docs.
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
> > > >> >>>> sewen@apache.org>
> > > >> >>>> > > wrote:
> > > >> >>>> > > > > > > > >
> > > >> >>>> > > > > > > > > @Andrey about the increase in metaspace size
> > > >> >>>> > > > > > > > > - I have no concerns for 1.11.0.
> > > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure,
> because
> > > >> users
> > > >> >>>> expect
> > > >> >>>> > > to
> > > >> >>>> > > > > > > upgrade
> > > >> >>>> > > > > > > > > that without config adjustments. That might not
> > be
> > > >> >>>> possible
> > > >> >>>> > > with
> > > >> >>>> > > > > that
> > > >> >>>> > > > > > > > > change.
> > > >> >>>> > > > > > > > >
> > > >> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey
> Zagrebin
> > <
> > > >> >>>> > > > > > > > azagrebin.apache@gmail.com>
> > > >> >>>> > > > > > > > > wrote:
> > > >> >>>> > > > > > > > >
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes
> Metaspace
> > > OOM
> > > >> >>>> error
> > > >> >>>> > on
> > > >> >>>> > > > > > repeated
> > > >> >>>> > > > > > > > >> job”
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> My understanding that the issue is basically
> > > covered
> > > >> >>>> by:
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should
> > be
> > > >> >>>> handled as
> > > >> >>>> > > > Fatal
> > > >> >>>> > > > > > > Error
> > > >> >>>> > > > > > > > >> in TaskManager
> > > >> >>>> > > > > > > > >> no full consensus there but improving error
> > > >> message
> > > >> >>>> for
> > > >> >>>> > > > existing
> > > >> >>>> > > > > > > task
> > > >> >>>> > > > > > > > >> thread fatal handling could be done at least
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
> > > >> >>>> Metaspace to
> > > >> >>>> > > > > minimise
> > > >> >>>> > > > > > > its
> > > >> >>>> > > > > > > > >> OutOfMemoryError
> > > >> >>>> > > > > > > > >> see further
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> - [FLINK-16246] Exclude
> > "SdkMBeanRegistrySupport"
> > > >> from
> > > >> >>>> > > > dynamically
> > > >> >>>> > > > > > > > loaded
> > > >> >>>> > > > > > > > >> AWS connectors
> > > >> >>>> > > > > > > > >> not sure whether this is a blocker but looks
> > > close
> > > >> to
> > > >> >>>> be
> > > >> >>>> > > > resolved
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for
> > JVM
> > > >> >>>> > Metaspace"
> > > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> > > bugfix
> > > >> >>>> release?
> > > >> >>>> > > It
> > > >> >>>> > > > > > > changes
> > > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> > > >> memory
> > > >> >>>> on
> > > >> >>>> > > > existing
> > > >> >>>> > > > > > > setups
> > > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> My understanding was that increasing to 256m
> > > >> resolved
> > > >> >>>> the
> > > >> >>>> > > > reported
> > > >> >>>> > > > > > > > problems
> > > >> >>>> > > > > > > > >> and we decided to make the change so I have
> > merged
> > > >> it
> > > >> >>>> today
> > > >> >>>> > as
> > > >> >>>> > > > > there
> > > >> >>>> > > > > > > > were
> > > >> >>>> > > > > > > > >> no more concerns.
> > > >> >>>> > > > > > > > >> If there are concerns I can revert it.
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> On the other hand, I think improving the
> message
> > > >> error
> > > >> >>>> with
> > > >> >>>> > > > > > reference
> > > >> >>>> > > > > > > to
> > > >> >>>> > > > > > > > >> the metaspace option should help the most
> > > >> >>>> > > > > > > > >> because user would not have to read all docs
> to
> > > fix
> > > >> it
> > > >> >>>> > > > > > > > >> then maybe this change is not even needed.
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >> Best,
> > > >> >>>> > > > > > > > >> Andrey
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> > > >> >>>> sewen@apache.org>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes
> Metaspace
> > > OOM
> > > >> >>>> error
> > > >> >>>> > on
> > > >> >>>> > > > > > repeated
> > > >> >>>> > > > > > > > >> job"
> > > >> >>>> > > > > > > > >>> - I don't think we have consensus on the
> exact
> > > >> >>>> solution,
> > > >> >>>> > yet,
> > > >> >>>> > > > and
> > > >> >>>> > > > > > > some
> > > >> >>>> > > > > > > > >> of
> > > >> >>>> > > > > > > > >>> the changes might also have side effects that
> > are
> > > >> >>>> hard to
> > > >> >>>> > > > > predict,
> > > >> >>>> > > > > > > so I
> > > >> >>>> > > > > > > > >> am
> > > >> >>>> > > > > > > > >>> not sure we should rush this in.
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for
> > JVM
> > > >> >>>> > Metaspace"
> > > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> > > bugfix
> > > >> >>>> release?
> > > >> >>>> > > It
> > > >> >>>> > > > > > > changes
> > > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> > > >> memory
> > > >> >>>> on
> > > >> >>>> > > > existing
> > > >> >>>> > > > > > > setups
> > > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>> We may need to unblock the release form these
> > two
> > > >> >>>> issues
> > > >> >>>> > and
> > > >> >>>> > > > > think
> > > >> >>>> > > > > > > > about
> > > >> >>>> > > > > > > > >>> having 1.10.2 in the near future.
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> > > >> >>>> carp84@gmail.com>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > > > > > >>>
> > > >> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an
> eye
> > > on
> > > >> >>>> these
> > > >> >>>> > two.
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>>> Best Regards,
> > > >> >>>> > > > > > > > >>>> Yu
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> > > >> >>>> imjark@gmail.com>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
> > > >> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>> From the Table SQL module, I think we
> should
> > > also
> > > >> >>>> try to
> > > >> >>>> > > get
> > > >> >>>> > > > in
> > > >> >>>> > > > > > the
> > > >> >>>> > > > > > > > >>>>> following issues:
> > > >> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override
> > > flink-conf
> > > >> >>>> > > parameters
> > > >> >>>> > > > > from
> > > >> >>>> > > > > > > SQL
> > > >> >>>> > > > > > > > >> CLI
> > > >> >>>> > > > > > > > >>>>> environment
> > > >> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
> > > >> >>>> watermark
> > > >> >>>> > > > interval,
> > > >> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > >> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any
> > > >> uncatched
> > > >> >>>> > > > exception
> > > >> >>>> > > > > is
> > > >> >>>> > > > > > > > >> thrown
> > > >> >>>> > > > > > > > >>>>> this will improve much experience when
> using
> > > SQL
> > > >> CLI
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>> Best,
> > > >> >>>> > > > > > > > >>>>> Jark
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> > > >> >>>> carp84@gmail.com>
> > > >> >>>> > > > wrote:
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've
> added
> > > >> >>>> 1.10.1 into
> > > >> >>>> > > > > > > FLINK-16225
> > > >> >>>> > > > > > > > >>>> fix
> > > >> >>>> > > > > > > > >>>>>> versions and promoted its priority to
> > > Critical.
> > > >> >>>> Will
> > > >> >>>> > also
> > > >> >>>> > > > > watch
> > > >> >>>> > > > > > > the
> > > >> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>> Best Regards,
> > > >> >>>> > > > > > > > >>>>>> Yu
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> > Zagrebin
> > > <
> > > >> >>>> > > > > > > azagrebin@apache.org
> > > >> >>>> > > > > > > > >
> > > >> >>>> > > > > > > > >>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>>> Hi Yu,
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
> > > >> >>>> discussion!
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> Apart from
> > > >> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for
> > JVM
> > > >> >>>> Metaspace
> > > >> >>>> > to
> > > >> >>>> > > > > > > minimise
> > > >> >>>> > > > > > > > >>>> its
> > > >> >>>> > > > > > > > >>>>>>> OutOfMemoryError
> > > >> >>>> > > > > > > > >>>>>>> which should be merged soon
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> I think we should also try to get in the
> > > >> following
> > > >> >>>> > > issues:
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> > > should
> > > >> be
> > > >> >>>> > handled
> > > >> >>>> > > > as
> > > >> >>>> > > > > > > Fatal
> > > >> >>>> > > > > > > > >>>>> Error
> > > >> >>>> > > > > > > > >>>>>> in
> > > >> >>>> > > > > > > > >>>>>>> TaskManager
> > > >> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem
> > even
> > > >> in a
> > > >> >>>> > better
> > > >> >>>> > > > way
> > > >> >>>> > > > > > > > because
> > > >> >>>> > > > > > > > >>>>> OOM
> > > >> >>>> > > > > > > > >>>>>>> failure should point users to the docs
> > > >> immediately
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class
> loader
> > > to
> > > >> >>>> lifetime
> > > >> >>>> > > of
> > > >> >>>> > > > a
> > > >> >>>> > > > > > slot
> > > >> >>>> > > > > > > > >>>>>>> This should give a better protection
> > against
> > > >> class
> > > >> >>>> > > loading
> > > >> >>>> > > > > > leaks
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting
> > when
> > > >> >>>> submitting
> > > >> >>>> > > > batch
> > > >> >>>> > > > > > job
> > > >> >>>> > > > > > > > >>>>>> (instead
> > > >> >>>> > > > > > > > >>>>>>> of AskTimeoutException)
> > > >> >>>> > > > > > > > >>>>>>> This problem has recently happened for
> > > multiple
> > > >> >>>> users
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>> Andrey
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> > Li <
> > > >> >>>> > > > > > > > jingsonglee0@gmail.com>
> > > >> >>>> > > > > > > > >>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting
> > the
> > > >> >>>> 1.10.1
> > > >> >>>> > > > release.
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users
> are
> > > >> looking
> > > >> >>>> > > forward
> > > >> >>>> > > > to
> > > >> >>>> > > > > > > them.
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>>> Jingsong Lee
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> > Guo <
> > > >> >>>> > > > > > karmagyz@gmail.com>
> > > >> >>>> > > > > > > > >>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release
> cycle.
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>>>> Yangze Guo
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> > > Song
> > > >> <
> > > >> >>>> > > > > > > > >>>>> tonysong820@gmail.com>
> > > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>> Yu,
> > > >> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
> > > >> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to
> > get
> > > >> some
> > > >> >>>> > inputs
> > > >> >>>> > > > for
> > > >> >>>> > > > > > > > >>>>>>> prioritizing
> > > >> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds
> > good
> > > to
> > > >> >>>> me.
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>> Thank you~
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>> Xintong Song
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu
> Li <
> > > >> >>>> > > > carp84@gmail.com>
> > > >> >>>> > > > > > > > >>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first
> > patch
> > > >> >>>> release
> > > >> >>>> > of
> > > >> >>>> > > a
> > > >> >>>> > > > > > major
> > > >> >>>> > > > > > > > >>>>>>> version
> > > >> >>>> > > > > > > > >>>>>>>>> will
> > > >> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
> > > >> >>>> depending on
> > > >> >>>> > > the
> > > >> >>>> > > > > > speed
> > > >> >>>> > > > > > > > >>>> of
> > > >> >>>> > > > > > > > >>>>>>>> blocker
> > > >> >>>> > > > > > > > >>>>>>>>>>> issue resolving:
> > > >> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May
> 28th
> > > >> [1],
> > > >> >>>> > released
> > > >> >>>> > > > on
> > > >> >>>> > > > > > Jul
> > > >> >>>> > > > > > > > >>>>> 3rd
> > > >> >>>> > > > > > > > >>>>>>> [2]
> > > >> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep
> 23rd
> > > >> [3],
> > > >> >>>> > released
> > > >> >>>> > > > on
> > > >> >>>> > > > > > Oct
> > > >> >>>> > > > > > > > >>>>> 19th
> > > >> >>>> > > > > > > > >>>>>>> [4]
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of
> > > >> course,
> > > >> >>>> but
> > > >> >>>> > > could
> > > >> >>>> > > > > use
> > > >> >>>> > > > > > > > >>>> it
> > > >> >>>> > > > > > > > >>>>>> as a
> > > >> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to
> let
> > me
> > > >> >>>> know if
> > > >> >>>> > any
> > > >> >>>> > > > > > > > >>>> concerns
> > > >> >>>> > > > > > > > >>>>>>>> Xintong.
> > > >> >>>> > > > > > > > >>>>>>>>>>> Thanks.
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> Best Regards,
> > > >> >>>> > > > > > > > >>>>>>>>>>> Yu
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> [1]
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > >
> > > >> >>>> > >
> > > >> >>>> >
> > > >> >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > >> >>>> > > > > > > > >>>>>>>>>>> [2]
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > >
> > > >> >>>> > >
> > > >> >>>> >
> > > >> >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > >> >>>> > > > > > > > >>>>>>>>>>> [3]
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > >
> > > >> >>>> > >
> > > >> >>>> >
> > > >> >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > >> >>>> > > > > > > > >>>>>>>>>>> [4]
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > >
> > > >> >>>> > >
> > > >> >>>> >
> > > >> >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> > > Song
> > > >> <
> > > >> >>>> > > > > > > > >>>>>> tonysong820@gmail.com>
> > > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and
> > > >> volunteering
> > > >> >>>> to be
> > > >> >>>> > > the
> > > >> >>>> > > > > > > > >>>>> release
> > > >> >>>> > > > > > > > >>>>>>>>> manager.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time
> > > plan
> > > >> for
> > > >> >>>> > > 1.10.1?
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> Thank you~
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> > > Zhijiang
> > > >> >>>> > > > > > > > >>>>>>>>>>>> <wangzhijiang999@aliyun.com
> .invalid>
> > > >> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release,
> Yu!
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > >
> > > >> >>>> >
> > > ------------------------------------------------------------------
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing
> Flink
> > > >> 1.10.1
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn.
> Yes
> > > will
> > > >> >>>> also
> > > >> >>>> > keep
> > > >> >>>> > > > an
> > > >> >>>> > > > > > > > >>>> eye
> > > >> >>>> > > > > > > > >>>>> on
> > > >> >>>> > > > > > > > >>>>>>>> these
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> Yu
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> > > >> Cheng <
> > > >> >>>> > > > > > > > >>>>> hequn@apache.org>
> > > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the
> > > discussion
> > > >> and
> > > >> >>>> > > > volunteer
> > > >> >>>> > > > > > > > >>>> as
> > > >> >>>> > > > > > > > >>>>>> the
> > > >> >>>> > > > > > > > >>>>>>>>> release
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> manager!
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other
> > issues[1]
> > > >> >>>> which are
> > > >> >>>> > > > > marked
> > > >> >>>> > > > > > > > >>>>> as
> > > >> >>>> > > > > > > > >>>>>> a
> > > >> >>>> > > > > > > > >>>>>>>>> blocker:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> > > year
> > > >> in
> > > >> >>>> > NOTICE
> > > >> >>>> > > > > files
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> > > with
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> > > >> >>>> and
> > > >> >>>> > > usrlib
> > > >> >>>> > > > > > > > >>>>>> directory
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170
> SearchTemplateRequest
> > > >> >>>> > > > > > > > >>>>> ClassNotFoundException
> > > >> >>>> > > > > > > > >>>>>>> when
> > > >> >>>> > > > > > > > >>>>>>>>> use
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error
> > reporting
> > > >> when
> > > >> >>>> > > > submitting
> > > >> >>>> > > > > > > > >>>>> batch
> > > >> >>>> > > > > > > > >>>>>>> job
> > > >> >>>> > > > > > > > >>>>>>>>>>>> (instead
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> of
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved
> > in
> > > >> >>>> 1.10.1.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> [1]
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > >
> > > >> >>>>
> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> > Li
> > > <
> > > >> >>>> > > > > > > > >>>> carp84@gmail.com>
> > > >> >>>> > > > > > > > >>>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very
> > helpful.
> > > >> >>>> Thanks a
> > > >> >>>> > > lot!
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> > > jincheng
> > > >> >>>> sun <
> > > >> >>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the
> discussion
> > > >> Yu. I
> > > >> >>>> would
> > > >> >>>> > > > like
> > > >> >>>> > > > > > > > >>>>> to
> > > >> >>>> > > > > > > > >>>>>>> give
> > > >> >>>> > > > > > > > >>>>>>>>> you a
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> hand
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> at
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
> > > >> >>>> finished.(If you
> > > >> >>>> > > > > > > > >>>> need)
> > > >> >>>> > > > > > > > >>>>>> :)
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
> > > >> 于2020年3月10日周二
> > > >> >>>> > > 下午5:49写道：
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month
> > since
> > > we
> > > >> >>>> > released
> > > >> >>>> > > > > > > > >>>>> Flink
> > > >> >>>> > > > > > > > >>>>>>>>> 1.10.0. We
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> already
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
> > > >> >>>> improvements/bugs
> > > >> >>>> > in
> > > >> >>>> > > > > > > > >>>> the
> > > >> >>>> > > > > > > > >>>>>>>>>>> release-1.10
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> branch,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the
> 1.10.1
> > > >> >>>> release
> > > >> >>>> > > cycle.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove
> the
> > > >> >>>> license and
> > > >> >>>> > > > > > > > >>>> notice
> > > >> >>>> > > > > > > > >>>>>>> file
> > > >> >>>> > > > > > > > >>>>>>>> in
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB
> > resource
> > > >> leak
> > > >> >>>> in
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
> > > should
> > > >> be
> > > >> >>>> known
> > > >> >>>> > > in
> > > >> >>>> > > > > > > > >>>>>>>>> HiveCatalog
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
> > > >> >>>> ArrayIndexOufOBoundsException in
> > > >> >>>> > > > > > > > >>>>>>>> TypeExtractor
> > > >> >>>> > > > > > > > >>>>>>>>>>> when
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample
> is
> > > >> >>>> failed if
> > > >> >>>> > > > > > > > >>>> running
> > > >> >>>> > > > > > > > >>>>>> in
> > > >> >>>> > > > > > > > >>>>>>>>> blink
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> planner
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
> > > constraints
> > > >> >>>> are not
> > > >> >>>> > > > > > > > >>>> reset
> > > >> >>>> > > > > > > > >>>>>> on
> > > >> >>>> > > > > > > > >>>>>>>> task
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> recovery
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> > > >> function
> > > >> >>>> using
> > > >> >>>> > > sql
> > > >> >>>> > > > > > > > >>>>>>> casuing
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL
> > validation
> > > >> >>>> failed
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the
> > following
> > > >> >>>> issues
> > > >> >>>> > > should
> > > >> >>>> > > > > > > > >>>> be
> > > >> >>>> > > > > > > > >>>>>>>> merged
> > > >> >>>> > > > > > > > >>>>>>>>>>> before
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the
> Metaspace
> > > OOM
> > > >> >>>> issue):
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak
> causes
> > > >> >>>> Metaspace
> > > >> >>>> > OOM
> > > >> >>>> > > > > > > > >>>>> error
> > > >> >>>> > > > > > > > >>>>>> on
> > > >> >>>> > > > > > > > >>>>>>>>>>> repeated
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> job
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
> > > value
> > > >> >>>> for JVM
> > > >> >>>> > > > > > > > >>>>>> Metaspace
> > > >> >>>> > > > > > > > >>>>>>> to
> > > >> >>>> > > > > > > > >>>>>>>>>>>> minimise
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> its
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
> > > produces
> > > >> >>>> wrong
> > > >> >>>> > > > > > > > >>>>> aggregate
> > > >> >>>> > > > > > > > >>>>>>>>> results
> > > >> >>>> > > > > > > > >>>>>>>>>>> with
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> state
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can
> > not
> > > >> >>>> extract
> > > >> >>>> > > > > > > > >>>> correct
> > > >> >>>> > > > > > > > >>>>>>>> unique
> > > >> >>>> > > > > > > > >>>>>>>>> key
> > > >> >>>> > > > > > > > >>>>>>>>>>>> for
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the
> release
> > > >> >>>> manager and
> > > >> >>>> > > kick
> > > >> >>>> > > > > > > > >>>>> off
> > > >> >>>> > > > > > > > >>>>>>> the
> > > >> >>>> > > > > > > > >>>>>>>>>>> release
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>> process
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged.
> > > What
> > > >> do
> > > >> >>>> you
> > > >> >>>> > > > > > > > >>>> think?
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or
> > > missing
> > > >> >>>> blocker
> > > >> >>>> > > > > > > > >>>> issues
> > > >> >>>> > > > > > > > >>>>>>> need
> > > >> >>>> > > > > > > > >>>>>>>>> to be
> > > >> >>>> > > > > > > > >>>>>>>>>>>>> fixed
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know.
> > Thanks.
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>> --
> > > >> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
> > > >> >>>> > > > > > > > >>>>>>>>
> > > >> >>>> > > > > > > > >>>>>>>
> > > >> >>>> > > > > > > > >>>>>>
> > > >> >>>> > > > > > > > >>>>>
> > > >> >>>> > > > > > > > >>>>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > > >>
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > > >
> > > >> >>>> > > > > > >
> > > >> >>>> > > > > >
> > > >> >>>> > > > >
> > > >> >>>> > > >
> > > >> >>>> > >
> > > >> >>>> >
> > > >> >>>>
> > > >> >>>
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Jark Wu <im...@gmail.com>.
+1 to have a 1.10.1 RC soon. It has been a long time since 1.10.0 is
released.
Best,
Jark
On Wed, 15 Apr 2020 at 16:10, Till Rohrmann <tr...@apache.org> wrote:
> Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
> lot for driving this effort Yu!
>
> Cheers,
> Till
>
> On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
>
> > Thanks Weike and all others for the efforts!
> >
> > Here comes the latest status, we are in good shape and plan to produce
> RC1
> > next week.
> >
> > * Blockers (1 left)
> > - [Closed] FLINK-16018 Improve error reporting when submitting batch
> job
> > (instead of AskTimeoutException)
> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> repeated
> > job submission
> > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> when
> > use flink-sql-connector-elasticsearch7
> > - [Closed] FLINK-16262 Class loader problem with
> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > minimise its OutOfMemoryError
> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> client
> > can retrieve JobResult
> > - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> > throws UnsupportedOperationException
> > - [Closed] FLINK-16626 Exception encountered when cancelling a job in
> > yarn per-job mode
> > - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when the
> > input column is of composite type
> > - [PR reviewed] FLINK-16576 State inconsistency on restore with memory
> > state backends
> >
> > * Critical (1 left)
> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> > with state clean up
> > - [Closed] FLINK-16070 Blink planner can not extract correct unique key
> > for UpsertStreamTableSink
> > - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
> be
> > handled as Fatal Error in TaskManager
> > - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
> > - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> of a
> > slot
> >
> > Please let me know if any concerns/comments. Thanks.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk>
> wrote:
> >
> > > Hi Yu,
> > >
> > > Thanks for your updates. I am still working on the fix for FLINK-16626
> > and
> > > it is expected to be completed by this Sunday after thorough testing.
> > >
> > > Sincerely,
> > > Weike
> > >
> > > On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> > >
> > >> Updates for 1.10.1 watched issues (we are in good progress and almost
> > >> there
> > >> to produce the first RC, thanks all for the efforts):
> > >>
> > >> * Blockers (3 left)
> > >> - [Closed] FLINK-16018 Improve error reporting when submitting batch
> > job
> > >> (instead of AskTimeoutException)
> > >> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >> repeated
> > >> job submission
> > >> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> > when
> > >> use flink-sql-connector-elasticsearch7
> > >> - [Closed] FLINK-16262 Class loader problem with
> > >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > >> minimise its OutOfMemoryError
> > >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >> - [PR reviewed] FLINK-16576 State inconsistency on restore with
> memory
> > >> state backends
> > >> - [Under Discussion] FLINK-16626 Exception encountered when
> > cancelling a
> > >> job in yarn per-job mode
> > >> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> > >> client
> > >> can retrieve JobResult
> > >> - [PR reviewed] FLINK-16913
> > ReadableConfigToConfigurationAdapter#getEnum
> > >> throws UnsupportedOperationException
> > >>
> > >> * Critical (1 left)
> > >> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership"
> error
> > >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> results
> > >> with state clean up
> > >> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> > key
> > >> for UpsertStreamTableSink
> > >> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory
> should
> > be
> > >> handled as Fatal Error in TaskManager
> > >> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> > of
> > >> a
> > >> slot
> > >>
> > >> Please let me know if you find any missing ones, thanks.
> > >>
> > >> Best Regards,
> > >> Yu
> > >>
> > >>
> > >> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> > >>
> > >> > Here comes the latest status of issues in 1.10.1 watch list:
> > >> >
> > >> > * Blockers (4 left)
> > >> > - [Under Discussion] FLINK-16018 Improve error reporting when
> > >> submitting
> > >> > batch job (instead of AskTimeoutException)
> > >> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > >> > repeated job submission
> > >> > - [Closed] FLINK-16170 SearchTemplateRequest
> ClassNotFoundException
> > >> when
> > >> > use flink-sql-connector-elasticsearch7
> > >> > - [PR Approved] FLINK-16262 Class loader problem with
> > >> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > >> > minimise its OutOfMemoryError
> > >> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >> > - [In Progress] FLINK-16576 State inconsistency on restore with
> > memory
> > >> > state backends
> > >> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> > >> MiniCluster
> > >> > before client can retrieve JobResult
> > >> >
> > >> > * Critical (2 left)
> > >> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > results
> > >> > with state clean up
> > >> > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> > >> > unique key for UpsertStreamTableSink
> > >> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should
> > be
> > >> > handled as Fatal Error in TaskManager
> > >> > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
> > slot
> > >> >
> > >> > There's a new blocker added. Please let me know if you find any
> other
> > >> > issues should be put into the watch list. Thanks.
> > >> >
> > >> > Best Regards,
> > >> > Yu
> > >> >
> > >> >
> > >> > On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> > >> >
> > >> >> Hi All,
> > >> >>
> > >> >> Here is the status update of issues in 1.10.1 watch list:
> > >> >>
> > >> >> * Blockers (7)
> > >> >>
> > >> >> - [Under Discussion] FLINK-16018 Improve error reporting when
> > >> >> submitting batch job (instead of AskTimeoutException)
> > >> >>
> > >> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
> > >> error
> > >> >> on repeated job submission
> > >> >>
> > >> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >> >> ClassNotFoundException when use flink-sql-connector-elasticsearch7
> > >> >>
> > >> >> - [PR Submitted] FLINK-16262 Class loader problem with
> > >> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >> >>
> > >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> to
> > >> >> minimise its OutOfMemoryError
> > >> >>
> > >> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > >> >>
> > >> >> - [Open] FLINK-16576 State inconsistency on restore with memory
> > state
> > >> >> backends
> > >> >>
> > >> >>
> > >> >> * Critical (4)
> > >> >>
> > >> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> > results
> > >> >> with state clean up
> > >> >>
> > >> >> - [PR Submitted] FLINK-16070 Blink planner can not extract
> correct
> > >> >> unique key for UpsertStreamTableSink
> > >> >>
> > >> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> be
> > >> >> handled as Fatal Error in TaskManager
> > >> >>
> > >> >> - [Open] FLINK-16408 Bind user code class loader to lifetime of a
> > >> slot
> > >> >>
> > >> >> Please let me know if you see any new blockers to add to the list.
> > >> Thanks.
> > >> >>
> > >> >> Best Regards,
> > >> >> Yu
> > >> >>
> > >> >>
> > >> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> > >> >>
> > >> >>> Thanks for the updates Till!
> > >> >>>
> > >> >>> For FLINK-16018, maybe we could create two sub-tasks for easy and
> > >> >>> complete fix separately, and only include the easy one in 1.10.1?
> Or
> > >> please
> > >> >>> just feel free to postpone the whole task to 1.10.2 if "all or
> > >> nothing"
> > >> >>> policy is preferred (smile). Thanks.
> > >> >>>
> > >> >>> Best Regards,
> > >> >>> Yu
> > >> >>>
> > >> >>>
> > >> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <trohrmann@apache.org
> >
> > >> >>> wrote:
> > >> >>>
> > >> >>>> +1 for a soonish bug fix release. Thanks for volunteering as our
> > >> release
> > >> >>>> manager Yu.
> > >> >>>>
> > >> >>>> I think we can soon merge the increase of metaspace size and
> > >> improving
> > >> >>>> the
> > >> >>>> error message. The assumption is that we currently don't have too
> > >> many
> > >> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of
> course,
> > >> the
> > >> >>>> sooner we release the bug fix release, the fewer deployments will
> > be
> > >> >>>> affected by this change.
> > >> >>>>
> > >> >>>> For FLINK-16018, I think there would be an easy solution which we
> > >> could
> > >> >>>> include in the bug fix release. The proper fix will most likely
> > take
> > >> a
> > >> >>>> bit
> > >> >>>> longer, though.
> > >> >>>>
> > >> >>>> Cheers,
> > >> >>>> Till
> > >> >>>>
> > >> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> > >> azagrebin@apache.org>
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>> > > @Andrey and @Xintong - could we have a quick poll on the user
> > >> >>>> mailing
> > >> >>>> > list
> > >> >>>> > > about increasing the metaspace size in Flink 1.10.1?
> > Specifically
> > >> >>>> asking
> > >> >>>> > > for who has very small TM setups?
> > >> >>>> >
> > >> >>>> > There has been a survey about this topic since 10 days:
> > >> >>>> >
> > >> >>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
> > >> >>>> > I can ask there about the small TM setups.
> > >> >>>> >
> > >> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com>
> wrote:
> > >> >>>> >
> > >> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency
> on
> > >> >>>> restore
> > >> >>>> > with
> > >> >>>> > > memory state backends
> > >> >>>> > >
> > >> >>>> > > Let me recompile the watching list with recent feedbacks.
> > >> There're
> > >> >>>> > totally
> > >> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of
> > >> which 14
> > >> >>>> > > already resolved and 31 left, and the below ones are watched
> > >> >>>> (meaning
> > >> >>>> > that
> > >> >>>> > > once the below ones got fixed, we will kick of the releasing
> > with
> > >> >>>> left
> > >> >>>> > ones
> > >> >>>> > > postponed to 1.10.2 unless objections):
> > >> >>>> > >
> > >> >>>> > > * Blockers (7)
> > >> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting
> when
> > >> >>>> > submitting
> > >> >>>> > > batch job (instead of AskTimeoutException)
> > >> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes
> Metaspace
> > >> OOM
> > >> >>>> error
> > >> >>>> > > on repeated job submission
> > >> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > >> >>>> > ClassNotFoundException
> > >> >>>> > > when use flink-sql-connector-elasticsearch7
> > >> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
> > >> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > >> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM
> > >> Metaspace to
> > >> >>>> > > minimise its OutOfMemoryError
> > >> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE
> > files
> > >> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with
> > memory
> > >> >>>> state
> > >> >>>> > > backends
> > >> >>>> > >
> > >> >>>> > > * Critical (4)
> > >> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate
> > >> >>>> results
> > >> >>>> > with
> > >> >>>> > > state clean up
> > >> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract
> > >> correct
> > >> >>>> > unique
> > >> >>>> > > key for UpsertStreamTableSink
> > >> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> > >> should be
> > >> >>>> > > handled as Fatal Error in TaskManager
> > >> >>>> > > - [Open] FLINK-16408 Bind user code class loader to
> lifetime
> > >> of a
> > >> >>>> slot
> > >> >>>> > >
> > >> >>>> > > Please raise your hand if you find any other issues should be
> > put
> > >> >>>> into
> > >> >>>> > this
> > >> >>>> > > list, thanks.
> > >> >>>> > >
> > >> >>>> > > Please also note that 1.10.2 version is already created
> (thanks
> > >> for
> > >> >>>> the
> > >> >>>> > > help @jincheng), and feel free to adjust/assign fix version
> to
> > it
> > >> >>>> when
> > >> >>>> > > necessary.
> > >> >>>> > >
> > >> >>>> > > Best Regards,
> > >> >>>> > > Yu
> > >> >>>> > >
> > >> >>>> > >
> > >> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <sewen@apache.org
> >
> > >> >>>> wrote:
> > >> >>>> > >
> > >> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the
> user
> > >> >>>> mailing
> > >> >>>> > > list
> > >> >>>> > > > about increasing the metaspace size in Flink 1.10.1?
> > >> Specifically
> > >> >>>> > asking
> > >> >>>> > > > for who has very small TM setups?
> > >> >>>> > > >
> > >> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> > >> wrote:
> > >> >>>> > > >
> > >> >>>> > > > > Thanks for the reminder Stephan and the inputs/discussion
> > >> >>>> Andrey and
> > >> >>>> > > > > Xintong. I totally agree that we should try best not to
> > >> >>>> introduce new
> > >> >>>> > > > > issues when resolving the existing one. Will watch
> closely
> > >> >>>> about the
> > >> >>>> > > > > discussion and conclusion around metaspace configuration.
> > >> >>>> > > > >
> > >> >>>> > > > > From the point as a developer instead of RM, I'm
> optimistic
> > >> that
> > >> >>>> > > > reserving
> > >> >>>> > > > > 160MB more memory to JVM metaspace by default won't
> cause a
> > >> big
> > >> >>>> > > problem,
> > >> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB
> > >> memory
> > >> >>>> per TM
> > >> >>>> > > is
> > >> >>>> > > > > the common case.
> > >> >>>> > > > >
> > >> >>>> > > > > Best Regards,
> > >> >>>> > > > > Yu
> > >> >>>> > > > >
> > >> >>>> > > > >
> > >> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> > >> >>>> tonysong820@gmail.com>
> > >> >>>> > > > wrote:
> > >> >>>> > > > >
> > >> >>>> > > > > > I think in FLINK-16406 we have not only increased the
> > >> default
> > >> >>>> value
> > >> >>>> > > for
> > >> >>>> > > > > > metaspace, but also increased the 'process.size' in
> > default
> > >> >>>> > > > > > 'flink-conf.yaml'.
> > >> >>>> > > > > >
> > >> >>>> > > > > > That means:
> > >> >>>> > > > > >
> > >> >>>> > > > > > - For people using the default 'flink-conf.yaml',
> it's
> > >> not
> > >> >>>> > really
> > >> >>>> > > > > taking
> > >> >>>> > > > > > memory from heap / managed for metaspace.
> > >> >>>> > > > > > - For people using custom configuration files:
> > >> >>>> > > > > > - If they are specifying 'flink.size', then
> > metaspace
> > >> >>>> does
> > >> >>>> > not
> > >> >>>> > > > > really
> > >> >>>> > > > > > affect the flink memory internal components
> (heap,
> > >> >>>> managed,
> > >> >>>> > > > etc.).
> > >> >>>> > > > > > - If they are specifying 'process.size' (most
> > likely
> > >> >>>> active
> > >> >>>> > > > > setups),
> > >> >>>> > > > > > then they will have less heap / managed memory,
> or
> > >> >>>> worse,
> > >> >>>> > their
> > >> >>>> > > > > > configuration may not work without adjustment.
> > >> >>>> > > > > > - For a TM with large process memory (e.g., 4
> GB
> > >> or
> > >> >>>> more),
> > >> >>>> > > it
> > >> >>>> > > > is
> > >> >>>> > > > > > probably ok that the metaspace takes 160 MB
> more
> > >> >>>> memory.
> > >> >>>> > > > > > - For a TM with smaller total memory (1 or 2
> GB)
> > >> the
> > >> >>>> > > metaspace
> > >> >>>> > > > > > increase is more likely to cause problem.
> > >> >>>> > > > > >
> > >> >>>> > > > > > So basically only people have small 'process.size' in
> > >> custom
> > >> >>>> config
> > >> >>>> > > > file
> > >> >>>> > > > > > are really affected. I'm not sure what is the
> proportion
> > of
> > >> >>>> such
> > >> >>>> > use
> > >> >>>> > > > > cases
> > >> >>>> > > > > > though. (From questions asked on the user ML, probably
> > not
> > >> >>>> much).
> > >> >>>> > > > > >
> > >> >>>> > > > > > Thank you~
> > >> >>>> > > > > >
> > >> >>>> > > > > > Xintong Song
> > >> >>>> > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> > >> >>>> sewen@apache.org>
> > >> >>>> > > > wrote:
> > >> >>>> > > > > >
> > >> >>>> > > > > > > No need to revert it now - I am not saying it should
> > not
> > >> go
> > >> >>>> into
> > >> >>>> > > > > 1.10.1,
> > >> >>>> > > > > > I
> > >> >>>> > > > > > > am just saying this is not clear to me yet.
> > >> >>>> > > > > > >
> > >> >>>> > > > > > > We have to trade off the fact that we may break some
> > >> >>>> deployments
> > >> >>>> > > with
> > >> >>>> > > > > the
> > >> >>>> > > > > > > fact that we will "safe" a lot of new deployments.
> > >> >>>> > > > > > > I simply lack the data points / insight at the moment
> > to
> > >> >>>> > understand
> > >> >>>> > > > how
> > >> >>>> > > > > > > significant both cases are, meaning how many users
> > would
> > >> be
> > >> >>>> > > affected
> > >> >>>> > > > > and
> > >> >>>> > > > > > > how badly.
> > >> >>>> > > > > > >
> > >> >>>> > > > > > > Independent of that, improving the error message is
> > >> always
> > >> >>>> > helpful.
> > >> >>>> > > > > > >
> > >> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > >> >>>> > > > > > > azagrebin.apache@gmail.com>
> > >> >>>> > > > > > > wrote:
> > >> >>>> > > > > > >
> > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
> > >> users
> > >> >>>> expect
> > >> >>>> > > to
> > >> >>>> > > > > > > upgrade
> > >> >>>> > > > > > > > > that without config adjustments. That might not
> be
> > >> >>>> possible
> > >> >>>> > > with
> > >> >>>> > > > > that
> > >> >>>> > > > > > > > > change.
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only
> > >> try to
> > >> >>>> > > improve
> > >> >>>> > > > > > error
> > >> >>>> > > > > > > > message and docs.
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
> > >> >>>> sewen@apache.org>
> > >> >>>> > > wrote:
> > >> >>>> > > > > > > > >
> > >> >>>> > > > > > > > > @Andrey about the increase in metaspace size
> > >> >>>> > > > > > > > > - I have no concerns for 1.11.0.
> > >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
> > >> users
> > >> >>>> expect
> > >> >>>> > > to
> > >> >>>> > > > > > > upgrade
> > >> >>>> > > > > > > > > that without config adjustments. That might not
> be
> > >> >>>> possible
> > >> >>>> > > with
> > >> >>>> > > > > that
> > >> >>>> > > > > > > > > change.
> > >> >>>> > > > > > > > >
> > >> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin
> <
> > >> >>>> > > > > > > > azagrebin.apache@gmail.com>
> > >> >>>> > > > > > > > > wrote:
> > >> >>>> > > > > > > > >
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace
> > OOM
> > >> >>>> error
> > >> >>>> > on
> > >> >>>> > > > > > repeated
> > >> >>>> > > > > > > > >> job”
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> My understanding that the issue is basically
> > covered
> > >> >>>> by:
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should
> be
> > >> >>>> handled as
> > >> >>>> > > > Fatal
> > >> >>>> > > > > > > Error
> > >> >>>> > > > > > > > >> in TaskManager
> > >> >>>> > > > > > > > >> no full consensus there but improving error
> > >> message
> > >> >>>> for
> > >> >>>> > > > existing
> > >> >>>> > > > > > > task
> > >> >>>> > > > > > > > >> thread fatal handling could be done at least
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
> > >> >>>> Metaspace to
> > >> >>>> > > > > minimise
> > >> >>>> > > > > > > its
> > >> >>>> > > > > > > > >> OutOfMemoryError
> > >> >>>> > > > > > > > >> see further
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> - [FLINK-16246] Exclude
> "SdkMBeanRegistrySupport"
> > >> from
> > >> >>>> > > > dynamically
> > >> >>>> > > > > > > > loaded
> > >> >>>> > > > > > > > >> AWS connectors
> > >> >>>> > > > > > > > >> not sure whether this is a blocker but looks
> > close
> > >> to
> > >> >>>> be
> > >> >>>> > > > resolved
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for
> JVM
> > >> >>>> > Metaspace"
> > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> > bugfix
> > >> >>>> release?
> > >> >>>> > > It
> > >> >>>> > > > > > > changes
> > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> > >> memory
> > >> >>>> on
> > >> >>>> > > > existing
> > >> >>>> > > > > > > setups
> > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> My understanding was that increasing to 256m
> > >> resolved
> > >> >>>> the
> > >> >>>> > > > reported
> > >> >>>> > > > > > > > problems
> > >> >>>> > > > > > > > >> and we decided to make the change so I have
> merged
> > >> it
> > >> >>>> today
> > >> >>>> > as
> > >> >>>> > > > > there
> > >> >>>> > > > > > > > were
> > >> >>>> > > > > > > > >> no more concerns.
> > >> >>>> > > > > > > > >> If there are concerns I can revert it.
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> On the other hand, I think improving the message
> > >> error
> > >> >>>> with
> > >> >>>> > > > > > reference
> > >> >>>> > > > > > > to
> > >> >>>> > > > > > > > >> the metaspace option should help the most
> > >> >>>> > > > > > > > >> because user would not have to read all docs to
> > fix
> > >> it
> > >> >>>> > > > > > > > >> then maybe this change is not even needed.
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >> Best,
> > >> >>>> > > > > > > > >> Andrey
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> > >> >>>> sewen@apache.org>
> > >> >>>> > > > wrote:
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace
> > OOM
> > >> >>>> error
> > >> >>>> > on
> > >> >>>> > > > > > repeated
> > >> >>>> > > > > > > > >> job"
> > >> >>>> > > > > > > > >>> - I don't think we have consensus on the exact
> > >> >>>> solution,
> > >> >>>> > yet,
> > >> >>>> > > > and
> > >> >>>> > > > > > > some
> > >> >>>> > > > > > > > >> of
> > >> >>>> > > > > > > > >>> the changes might also have side effects that
> are
> > >> >>>> hard to
> > >> >>>> > > > > predict,
> > >> >>>> > > > > > > so I
> > >> >>>> > > > > > > > >> am
> > >> >>>> > > > > > > > >>> not sure we should rush this in.
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for
> JVM
> > >> >>>> > Metaspace"
> > >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> > bugfix
> > >> >>>> release?
> > >> >>>> > > It
> > >> >>>> > > > > > > changes
> > >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> > >> memory
> > >> >>>> on
> > >> >>>> > > > existing
> > >> >>>> > > > > > > setups
> > >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>> We may need to unblock the release form these
> two
> > >> >>>> issues
> > >> >>>> > and
> > >> >>>> > > > > think
> > >> >>>> > > > > > > > about
> > >> >>>> > > > > > > > >>> having 1.10.2 in the near future.
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> > >> >>>> carp84@gmail.com>
> > >> >>>> > > > wrote:
> > >> >>>> > > > > > > > >>>
> > >> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye
> > on
> > >> >>>> these
> > >> >>>> > two.
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>>> Best Regards,
> > >> >>>> > > > > > > > >>>> Yu
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> > >> >>>> imjark@gmail.com>
> > >> >>>> > > > wrote:
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
> > >> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>> From the Table SQL module, I think we should
> > also
> > >> >>>> try to
> > >> >>>> > > get
> > >> >>>> > > > in
> > >> >>>> > > > > > the
> > >> >>>> > > > > > > > >>>>> following issues:
> > >> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override
> > flink-conf
> > >> >>>> > > parameters
> > >> >>>> > > > > from
> > >> >>>> > > > > > > SQL
> > >> >>>> > > > > > > > >> CLI
> > >> >>>> > > > > > > > >>>>> environment
> > >> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
> > >> >>>> watermark
> > >> >>>> > > > interval,
> > >> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > >> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any
> > >> uncatched
> > >> >>>> > > > exception
> > >> >>>> > > > > is
> > >> >>>> > > > > > > > >> thrown
> > >> >>>> > > > > > > > >>>>> this will improve much experience when using
> > SQL
> > >> CLI
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>> Best,
> > >> >>>> > > > > > > > >>>>> Jark
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> > >> >>>> carp84@gmail.com>
> > >> >>>> > > > wrote:
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added
> > >> >>>> 1.10.1 into
> > >> >>>> > > > > > > FLINK-16225
> > >> >>>> > > > > > > > >>>> fix
> > >> >>>> > > > > > > > >>>>>> versions and promoted its priority to
> > Critical.
> > >> >>>> Will
> > >> >>>> > also
> > >> >>>> > > > > watch
> > >> >>>> > > > > > > the
> > >> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>> Best Regards,
> > >> >>>> > > > > > > > >>>>>> Yu
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey
> Zagrebin
> > <
> > >> >>>> > > > > > > azagrebin@apache.org
> > >> >>>> > > > > > > > >
> > >> >>>> > > > > > > > >>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>>> Hi Yu,
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
> > >> >>>> discussion!
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> Apart from
> > >> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for
> JVM
> > >> >>>> Metaspace
> > >> >>>> > to
> > >> >>>> > > > > > > minimise
> > >> >>>> > > > > > > > >>>> its
> > >> >>>> > > > > > > > >>>>>>> OutOfMemoryError
> > >> >>>> > > > > > > > >>>>>>> which should be merged soon
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> I think we should also try to get in the
> > >> following
> > >> >>>> > > issues:
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> > should
> > >> be
> > >> >>>> > handled
> > >> >>>> > > > as
> > >> >>>> > > > > > > Fatal
> > >> >>>> > > > > > > > >>>>> Error
> > >> >>>> > > > > > > > >>>>>> in
> > >> >>>> > > > > > > > >>>>>>> TaskManager
> > >> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem
> even
> > >> in a
> > >> >>>> > better
> > >> >>>> > > > way
> > >> >>>> > > > > > > > because
> > >> >>>> > > > > > > > >>>>> OOM
> > >> >>>> > > > > > > > >>>>>>> failure should point users to the docs
> > >> immediately
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader
> > to
> > >> >>>> lifetime
> > >> >>>> > > of
> > >> >>>> > > > a
> > >> >>>> > > > > > slot
> > >> >>>> > > > > > > > >>>>>>> This should give a better protection
> against
> > >> class
> > >> >>>> > > loading
> > >> >>>> > > > > > leaks
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting
> when
> > >> >>>> submitting
> > >> >>>> > > > batch
> > >> >>>> > > > > > job
> > >> >>>> > > > > > > > >>>>>> (instead
> > >> >>>> > > > > > > > >>>>>>> of AskTimeoutException)
> > >> >>>> > > > > > > > >>>>>>> This problem has recently happened for
> > multiple
> > >> >>>> users
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>> Andrey
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong
> Li <
> > >> >>>> > > > > > > > jingsonglee0@gmail.com>
> > >> >>>> > > > > > > > >>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting
> the
> > >> >>>> 1.10.1
> > >> >>>> > > > release.
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users are
> > >> looking
> > >> >>>> > > forward
> > >> >>>> > > > to
> > >> >>>> > > > > > > them.
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>>> Jingsong Lee
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze
> Guo <
> > >> >>>> > > > > > karmagyz@gmail.com>
> > >> >>>> > > > > > > > >>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>>>> Yangze Guo
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> > Song
> > >> <
> > >> >>>> > > > > > > > >>>>> tonysong820@gmail.com>
> > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>> Yu,
> > >> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
> > >> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to
> get
> > >> some
> > >> >>>> > inputs
> > >> >>>> > > > for
> > >> >>>> > > > > > > > >>>>>>> prioritizing
> > >> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds
> good
> > to
> > >> >>>> me.
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>> Thank you~
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>> Xintong Song
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> > >> >>>> > > > carp84@gmail.com>
> > >> >>>> > > > > > > > >>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first
> patch
> > >> >>>> release
> > >> >>>> > of
> > >> >>>> > > a
> > >> >>>> > > > > > major
> > >> >>>> > > > > > > > >>>>>>> version
> > >> >>>> > > > > > > > >>>>>>>>> will
> > >> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
> > >> >>>> depending on
> > >> >>>> > > the
> > >> >>>> > > > > > speed
> > >> >>>> > > > > > > > >>>> of
> > >> >>>> > > > > > > > >>>>>>>> blocker
> > >> >>>> > > > > > > > >>>>>>>>>>> issue resolving:
> > >> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th
> > >> [1],
> > >> >>>> > released
> > >> >>>> > > > on
> > >> >>>> > > > > > Jul
> > >> >>>> > > > > > > > >>>>> 3rd
> > >> >>>> > > > > > > > >>>>>>> [2]
> > >> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd
> > >> [3],
> > >> >>>> > released
> > >> >>>> > > > on
> > >> >>>> > > > > > Oct
> > >> >>>> > > > > > > > >>>>> 19th
> > >> >>>> > > > > > > > >>>>>>> [4]
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of
> > >> course,
> > >> >>>> but
> > >> >>>> > > could
> > >> >>>> > > > > use
> > >> >>>> > > > > > > > >>>> it
> > >> >>>> > > > > > > > >>>>>> as a
> > >> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let
> me
> > >> >>>> know if
> > >> >>>> > any
> > >> >>>> > > > > > > > >>>> concerns
> > >> >>>> > > > > > > > >>>>>>>> Xintong.
> > >> >>>> > > > > > > > >>>>>>>>>>> Thanks.
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> Best Regards,
> > >> >>>> > > > > > > > >>>>>>>>>>> Yu
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> [1]
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > >
> > >> >>>> > > >
> > >> >>>> > >
> > >> >>>> >
> > >> >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > >> >>>> > > > > > > > >>>>>>>>>>> [2]
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > >
> > >> >>>> > > >
> > >> >>>> > >
> > >> >>>> >
> > >> >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > >> >>>> > > > > > > > >>>>>>>>>>> [3]
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > >
> > >> >>>> > > >
> > >> >>>> > >
> > >> >>>> >
> > >> >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > >> >>>> > > > > > > > >>>>>>>>>>> [4]
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > >
> > >> >>>> > > >
> > >> >>>> > >
> > >> >>>> >
> > >> >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> > Song
> > >> <
> > >> >>>> > > > > > > > >>>>>> tonysong820@gmail.com>
> > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and
> > >> volunteering
> > >> >>>> to be
> > >> >>>> > > the
> > >> >>>> > > > > > > > >>>>> release
> > >> >>>> > > > > > > > >>>>>>>>> manager.
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time
> > plan
> > >> for
> > >> >>>> > > 1.10.1?
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> Thank you~
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> > Zhijiang
> > >> >>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid>
> > >> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > >
> > >> >>>> >
> > ------------------------------------------------------------------
> > >> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > >> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink
> > >> 1.10.1
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes
> > will
> > >> >>>> also
> > >> >>>> > keep
> > >> >>>> > > > an
> > >> >>>> > > > > > > > >>>> eye
> > >> >>>> > > > > > > > >>>>> on
> > >> >>>> > > > > > > > >>>>>>>> these
> > >> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
> > >> >>>> > > > > > > > >>>>>>>>>>>>> Yu
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> > >> Cheng <
> > >> >>>> > > > > > > > >>>>> hequn@apache.org>
> > >> >>>> > > > > > > > >>>>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the
> > discussion
> > >> and
> > >> >>>> > > > volunteer
> > >> >>>> > > > > > > > >>>> as
> > >> >>>> > > > > > > > >>>>>> the
> > >> >>>> > > > > > > > >>>>>>>>> release
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> manager!
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other
> issues[1]
> > >> >>>> which are
> > >> >>>> > > > > marked
> > >> >>>> > > > > > > > >>>>> as
> > >> >>>> > > > > > > > >>>>>> a
> > >> >>>> > > > > > > > >>>>>>>>> blocker:
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> > year
> > >> in
> > >> >>>> > NOTICE
> > >> >>>> > > > > files
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> > with
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> > >> >>>> and
> > >> >>>> > > usrlib
> > >> >>>> > > > > > > > >>>>>> directory
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > >> >>>> > > > > > > > >>>>> ClassNotFoundException
> > >> >>>> > > > > > > > >>>>>>> when
> > >> >>>> > > > > > > > >>>>>>>>> use
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error
> reporting
> > >> when
> > >> >>>> > > > submitting
> > >> >>>> > > > > > > > >>>>> batch
> > >> >>>> > > > > > > > >>>>>>> job
> > >> >>>> > > > > > > > >>>>>>>>>>>> (instead
> > >> >>>> > > > > > > > >>>>>>>>>>>>> of
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved
> in
> > >> >>>> 1.10.1.
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> [1]
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > >
> > >> >>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu
> Li
> > <
> > >> >>>> > > > > > > > >>>> carp84@gmail.com>
> > >> >>>> > > > > > > > >>>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very
> helpful.
> > >> >>>> Thanks a
> > >> >>>> > > lot!
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> > jincheng
> > >> >>>> sun <
> > >> >>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion
> > >> Yu. I
> > >> >>>> would
> > >> >>>> > > > like
> > >> >>>> > > > > > > > >>>>> to
> > >> >>>> > > > > > > > >>>>>>> give
> > >> >>>> > > > > > > > >>>>>>>>> you a
> > >> >>>> > > > > > > > >>>>>>>>>>>>> hand
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> at
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
> > >> >>>> finished.(If you
> > >> >>>> > > > > > > > >>>> need)
> > >> >>>> > > > > > > > >>>>>> :)
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
> > >> 于2020年3月10日周二
> > >> >>>> > > 下午5:49写道：
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month
> since
> > we
> > >> >>>> > released
> > >> >>>> > > > > > > > >>>>> Flink
> > >> >>>> > > > > > > > >>>>>>>>> 1.10.0. We
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> already
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
> > >> >>>> improvements/bugs
> > >> >>>> > in
> > >> >>>> > > > > > > > >>>> the
> > >> >>>> > > > > > > > >>>>>>>>>>> release-1.10
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> branch,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1
> > >> >>>> release
> > >> >>>> > > cycle.
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the
> > >> >>>> license and
> > >> >>>> > > > > > > > >>>> notice
> > >> >>>> > > > > > > > >>>>>>> file
> > >> >>>> > > > > > > > >>>>>>>> in
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB
> resource
> > >> leak
> > >> >>>> in
> > >> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
> > should
> > >> be
> > >> >>>> known
> > >> >>>> > > in
> > >> >>>> > > > > > > > >>>>>>>>> HiveCatalog
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
> > >> >>>> ArrayIndexOufOBoundsException in
> > >> >>>> > > > > > > > >>>>>>>> TypeExtractor
> > >> >>>> > > > > > > > >>>>>>>>>>> when
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is
> > >> >>>> failed if
> > >> >>>> > > > > > > > >>>> running
> > >> >>>> > > > > > > > >>>>>> in
> > >> >>>> > > > > > > > >>>>>>>>> blink
> > >> >>>> > > > > > > > >>>>>>>>>>>>> planner
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
> > constraints
> > >> >>>> are not
> > >> >>>> > > > > > > > >>>> reset
> > >> >>>> > > > > > > > >>>>>> on
> > >> >>>> > > > > > > > >>>>>>>> task
> > >> >>>> > > > > > > > >>>>>>>>>>>>> recovery
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> > >> function
> > >> >>>> using
> > >> >>>> > > sql
> > >> >>>> > > > > > > > >>>>>>> casuing
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL
> validation
> > >> >>>> failed
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the
> following
> > >> >>>> issues
> > >> >>>> > > should
> > >> >>>> > > > > > > > >>>> be
> > >> >>>> > > > > > > > >>>>>>>> merged
> > >> >>>> > > > > > > > >>>>>>>>>>> before
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace
> > OOM
> > >> >>>> issue):
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
> > >> >>>> Metaspace
> > >> >>>> > OOM
> > >> >>>> > > > > > > > >>>>> error
> > >> >>>> > > > > > > > >>>>>> on
> > >> >>>> > > > > > > > >>>>>>>>>>> repeated
> > >> >>>> > > > > > > > >>>>>>>>>>>>> job
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
> > value
> > >> >>>> for JVM
> > >> >>>> > > > > > > > >>>>>> Metaspace
> > >> >>>> > > > > > > > >>>>>>> to
> > >> >>>> > > > > > > > >>>>>>>>>>>> minimise
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> its
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
> > produces
> > >> >>>> wrong
> > >> >>>> > > > > > > > >>>>> aggregate
> > >> >>>> > > > > > > > >>>>>>>>> results
> > >> >>>> > > > > > > > >>>>>>>>>>> with
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> state
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can
> not
> > >> >>>> extract
> > >> >>>> > > > > > > > >>>> correct
> > >> >>>> > > > > > > > >>>>>>>> unique
> > >> >>>> > > > > > > > >>>>>>>>> key
> > >> >>>> > > > > > > > >>>>>>>>>>>> for
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release
> > >> >>>> manager and
> > >> >>>> > > kick
> > >> >>>> > > > > > > > >>>>> off
> > >> >>>> > > > > > > > >>>>>>> the
> > >> >>>> > > > > > > > >>>>>>>>>>> release
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>> process
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged.
> > What
> > >> do
> > >> >>>> you
> > >> >>>> > > > > > > > >>>> think?
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or
> > missing
> > >> >>>> blocker
> > >> >>>> > > > > > > > >>>> issues
> > >> >>>> > > > > > > > >>>>>>> need
> > >> >>>> > > > > > > > >>>>>>>>> to be
> > >> >>>> > > > > > > > >>>>>>>>>>>>> fixed
> > >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know.
> Thanks.
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>> --
> > >> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
> > >> >>>> > > > > > > > >>>>>>>>
> > >> >>>> > > > > > > > >>>>>>>
> > >> >>>> > > > > > > > >>>>>>
> > >> >>>> > > > > > > > >>>>>
> > >> >>>> > > > > > > > >>>>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > > >>
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > > >
> > >> >>>> > > > > > >
> > >> >>>> > > > > >
> > >> >>>> > > > >
> > >> >>>> > > >
> > >> >>>> > >
> > >> >>>> >
> > >> >>>>
> > >> >>>
> > >>
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Till Rohrmann <tr...@apache.org>.
Great to see that will have the first RC for Flink 1.10.1 soon. Thanks a
lot for driving this effort Yu!
Cheers,
Till
On Sun, Apr 12, 2020 at 5:03 PM Yu Li <ca...@gmail.com> wrote:
> Thanks Weike and all others for the efforts!
>
> Here comes the latest status, we are in good shape and plan to produce RC1
> next week.
>
> * Blockers (1 left)
> - [Closed] FLINK-16018 Improve error reporting when submitting batch job
> (instead of AskTimeoutException)
> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> job submission
> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
> use flink-sql-connector-elasticsearch7
> - [Closed] FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> minimise its OutOfMemoryError
> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before client
> can retrieve JobResult
> - [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> throws UnsupportedOperationException
> - [Closed] FLINK-16626 Exception encountered when cancelling a job in
> yarn per-job mode
> - [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when the
> input column is of composite type
> - [PR reviewed] FLINK-16576 State inconsistency on restore with memory
> state backends
>
> * Critical (1 left)
> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> with state clean up
> - [Closed] FLINK-16070 Blink planner can not extract correct unique key
> for UpsertStreamTableSink
> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be
> handled as Fatal Error in TaskManager
> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime of a
> slot
>
> Please let me know if any concerns/comments. Thanks.
>
> Best Regards,
> Yu
>
>
> On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk> wrote:
>
> > Hi Yu,
> >
> > Thanks for your updates. I am still working on the fix for FLINK-16626
> and
> > it is expected to be completed by this Sunday after thorough testing.
> >
> > Sincerely,
> > Weike
> >
> > On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> >
> >> Updates for 1.10.1 watched issues (we are in good progress and almost
> >> there
> >> to produce the first RC, thanks all for the efforts):
> >>
> >> * Blockers (3 left)
> >> - [Closed] FLINK-16018 Improve error reporting when submitting batch
> job
> >> (instead of AskTimeoutException)
> >> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >> repeated
> >> job submission
> >> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> when
> >> use flink-sql-connector-elasticsearch7
> >> - [Closed] FLINK-16262 Class loader problem with
> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >> minimise its OutOfMemoryError
> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >> - [PR reviewed] FLINK-16576 State inconsistency on restore with memory
> >> state backends
> >> - [Under Discussion] FLINK-16626 Exception encountered when
> cancelling a
> >> job in yarn per-job mode
> >> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
> >> client
> >> can retrieve JobResult
> >> - [PR reviewed] FLINK-16913
> ReadableConfigToConfigurationAdapter#getEnum
> >> throws UnsupportedOperationException
> >>
> >> * Critical (1 left)
> >> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> >> with state clean up
> >> - [Closed] FLINK-16070 Blink planner can not extract correct unique
> key
> >> for UpsertStreamTableSink
> >> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should
> be
> >> handled as Fatal Error in TaskManager
> >> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime
> of
> >> a
> >> slot
> >>
> >> Please let me know if you find any missing ones, thanks.
> >>
> >> Best Regards,
> >> Yu
> >>
> >>
> >> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> >>
> >> > Here comes the latest status of issues in 1.10.1 watch list:
> >> >
> >> > * Blockers (4 left)
> >> > - [Under Discussion] FLINK-16018 Improve error reporting when
> >> submitting
> >> > batch job (instead of AskTimeoutException)
> >> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> >> > repeated job submission
> >> > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> >> when
> >> > use flink-sql-connector-elasticsearch7
> >> > - [PR Approved] FLINK-16262 Class loader problem with
> >> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >> > minimise its OutOfMemoryError
> >> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >> > - [In Progress] FLINK-16576 State inconsistency on restore with
> memory
> >> > state backends
> >> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
> >> MiniCluster
> >> > before client can retrieve JobResult
> >> >
> >> > * Critical (2 left)
> >> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> results
> >> > with state clean up
> >> > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> >> > unique key for UpsertStreamTableSink
> >> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should
> be
> >> > handled as Fatal Error in TaskManager
> >> > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
> slot
> >> >
> >> > There's a new blocker added. Please let me know if you find any other
> >> > issues should be put into the watch list. Thanks.
> >> >
> >> > Best Regards,
> >> > Yu
> >> >
> >> >
> >> > On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> >> >
> >> >> Hi All,
> >> >>
> >> >> Here is the status update of issues in 1.10.1 watch list:
> >> >>
> >> >> * Blockers (7)
> >> >>
> >> >> - [Under Discussion] FLINK-16018 Improve error reporting when
> >> >> submitting batch job (instead of AskTimeoutException)
> >> >>
> >> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
> >> error
> >> >> on repeated job submission
> >> >>
> >> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >> >> ClassNotFoundException when use flink-sql-connector-elasticsearch7
> >> >>
> >> >> - [PR Submitted] FLINK-16262 Class loader problem with
> >> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >> >>
> >> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >> >> minimise its OutOfMemoryError
> >> >>
> >> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >> >>
> >> >> - [Open] FLINK-16576 State inconsistency on restore with memory
> state
> >> >> backends
> >> >>
> >> >>
> >> >> * Critical (4)
> >> >>
> >> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate
> results
> >> >> with state clean up
> >> >>
> >> >> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> >> >> unique key for UpsertStreamTableSink
> >> >>
> >> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> >> >> handled as Fatal Error in TaskManager
> >> >>
> >> >> - [Open] FLINK-16408 Bind user code class loader to lifetime of a
> >> slot
> >> >>
> >> >> Please let me know if you see any new blockers to add to the list.
> >> Thanks.
> >> >>
> >> >> Best Regards,
> >> >> Yu
> >> >>
> >> >>
> >> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> >> >>
> >> >>> Thanks for the updates Till!
> >> >>>
> >> >>> For FLINK-16018, maybe we could create two sub-tasks for easy and
> >> >>> complete fix separately, and only include the easy one in 1.10.1? Or
> >> please
> >> >>> just feel free to postpone the whole task to 1.10.2 if "all or
> >> nothing"
> >> >>> policy is preferred (smile). Thanks.
> >> >>>
> >> >>> Best Regards,
> >> >>> Yu
> >> >>>
> >> >>>
> >> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org>
> >> >>> wrote:
> >> >>>
> >> >>>> +1 for a soonish bug fix release. Thanks for volunteering as our
> >> release
> >> >>>> manager Yu.
> >> >>>>
> >> >>>> I think we can soon merge the increase of metaspace size and
> >> improving
> >> >>>> the
> >> >>>> error message. The assumption is that we currently don't have too
> >> many
> >> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of course,
> >> the
> >> >>>> sooner we release the bug fix release, the fewer deployments will
> be
> >> >>>> affected by this change.
> >> >>>>
> >> >>>> For FLINK-16018, I think there would be an easy solution which we
> >> could
> >> >>>> include in the bug fix release. The proper fix will most likely
> take
> >> a
> >> >>>> bit
> >> >>>> longer, though.
> >> >>>>
> >> >>>> Cheers,
> >> >>>> Till
> >> >>>>
> >> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
> >> azagrebin@apache.org>
> >> >>>> wrote:
> >> >>>>
> >> >>>> > > @Andrey and @Xintong - could we have a quick poll on the user
> >> >>>> mailing
> >> >>>> > list
> >> >>>> > > about increasing the metaspace size in Flink 1.10.1?
> Specifically
> >> >>>> asking
> >> >>>> > > for who has very small TM setups?
> >> >>>> >
> >> >>>> > There has been a survey about this topic since 10 days:
> >> >>>> >
> >> >>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
> >> >>>> > I can ask there about the small TM setups.
> >> >>>> >
> >> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
> >> >>>> >
> >> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on
> >> >>>> restore
> >> >>>> > with
> >> >>>> > > memory state backends
> >> >>>> > >
> >> >>>> > > Let me recompile the watching list with recent feedbacks.
> >> There're
> >> >>>> > totally
> >> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of
> >> which 14
> >> >>>> > > already resolved and 31 left, and the below ones are watched
> >> >>>> (meaning
> >> >>>> > that
> >> >>>> > > once the below ones got fixed, we will kick of the releasing
> with
> >> >>>> left
> >> >>>> > ones
> >> >>>> > > postponed to 1.10.2 unless objections):
> >> >>>> > >
> >> >>>> > > * Blockers (7)
> >> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
> >> >>>> > submitting
> >> >>>> > > batch job (instead of AskTimeoutException)
> >> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> >> OOM
> >> >>>> error
> >> >>>> > > on repeated job submission
> >> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >> >>>> > ClassNotFoundException
> >> >>>> > > when use flink-sql-connector-elasticsearch7
> >> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
> >> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM
> >> Metaspace to
> >> >>>> > > minimise its OutOfMemoryError
> >> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE
> files
> >> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with
> memory
> >> >>>> state
> >> >>>> > > backends
> >> >>>> > >
> >> >>>> > > * Critical (4)
> >> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate
> >> >>>> results
> >> >>>> > with
> >> >>>> > > state clean up
> >> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract
> >> correct
> >> >>>> > unique
> >> >>>> > > key for UpsertStreamTableSink
> >> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
> >> should be
> >> >>>> > > handled as Fatal Error in TaskManager
> >> >>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime
> >> of a
> >> >>>> slot
> >> >>>> > >
> >> >>>> > > Please raise your hand if you find any other issues should be
> put
> >> >>>> into
> >> >>>> > this
> >> >>>> > > list, thanks.
> >> >>>> > >
> >> >>>> > > Please also note that 1.10.2 version is already created (thanks
> >> for
> >> >>>> the
> >> >>>> > > help @jincheng), and feel free to adjust/assign fix version to
> it
> >> >>>> when
> >> >>>> > > necessary.
> >> >>>> > >
> >> >>>> > > Best Regards,
> >> >>>> > > Yu
> >> >>>> > >
> >> >>>> > >
> >> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org>
> >> >>>> wrote:
> >> >>>> > >
> >> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the user
> >> >>>> mailing
> >> >>>> > > list
> >> >>>> > > > about increasing the metaspace size in Flink 1.10.1?
> >> Specifically
> >> >>>> > asking
> >> >>>> > > > for who has very small TM setups?
> >> >>>> > > >
> >> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
> >> wrote:
> >> >>>> > > >
> >> >>>> > > > > Thanks for the reminder Stephan and the inputs/discussion
> >> >>>> Andrey and
> >> >>>> > > > > Xintong. I totally agree that we should try best not to
> >> >>>> introduce new
> >> >>>> > > > > issues when resolving the existing one. Will watch closely
> >> >>>> about the
> >> >>>> > > > > discussion and conclusion around metaspace configuration.
> >> >>>> > > > >
> >> >>>> > > > > From the point as a developer instead of RM, I'm optimistic
> >> that
> >> >>>> > > > reserving
> >> >>>> > > > > 160MB more memory to JVM metaspace by default won't cause a
> >> big
> >> >>>> > > problem,
> >> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB
> >> memory
> >> >>>> per TM
> >> >>>> > > is
> >> >>>> > > > > the common case.
> >> >>>> > > > >
> >> >>>> > > > > Best Regards,
> >> >>>> > > > > Yu
> >> >>>> > > > >
> >> >>>> > > > >
> >> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> >> >>>> tonysong820@gmail.com>
> >> >>>> > > > wrote:
> >> >>>> > > > >
> >> >>>> > > > > > I think in FLINK-16406 we have not only increased the
> >> default
> >> >>>> value
> >> >>>> > > for
> >> >>>> > > > > > metaspace, but also increased the 'process.size' in
> default
> >> >>>> > > > > > 'flink-conf.yaml'.
> >> >>>> > > > > >
> >> >>>> > > > > > That means:
> >> >>>> > > > > >
> >> >>>> > > > > > - For people using the default 'flink-conf.yaml', it's
> >> not
> >> >>>> > really
> >> >>>> > > > > taking
> >> >>>> > > > > > memory from heap / managed for metaspace.
> >> >>>> > > > > > - For people using custom configuration files:
> >> >>>> > > > > > - If they are specifying 'flink.size', then
> metaspace
> >> >>>> does
> >> >>>> > not
> >> >>>> > > > > really
> >> >>>> > > > > > affect the flink memory internal components (heap,
> >> >>>> managed,
> >> >>>> > > > etc.).
> >> >>>> > > > > > - If they are specifying 'process.size' (most
> likely
> >> >>>> active
> >> >>>> > > > > setups),
> >> >>>> > > > > > then they will have less heap / managed memory, or
> >> >>>> worse,
> >> >>>> > their
> >> >>>> > > > > > configuration may not work without adjustment.
> >> >>>> > > > > > - For a TM with large process memory (e.g., 4 GB
> >> or
> >> >>>> more),
> >> >>>> > > it
> >> >>>> > > > is
> >> >>>> > > > > > probably ok that the metaspace takes 160 MB more
> >> >>>> memory.
> >> >>>> > > > > > - For a TM with smaller total memory (1 or 2 GB)
> >> the
> >> >>>> > > metaspace
> >> >>>> > > > > > increase is more likely to cause problem.
> >> >>>> > > > > >
> >> >>>> > > > > > So basically only people have small 'process.size' in
> >> custom
> >> >>>> config
> >> >>>> > > > file
> >> >>>> > > > > > are really affected. I'm not sure what is the proportion
> of
> >> >>>> such
> >> >>>> > use
> >> >>>> > > > > cases
> >> >>>> > > > > > though. (From questions asked on the user ML, probably
> not
> >> >>>> much).
> >> >>>> > > > > >
> >> >>>> > > > > > Thank you~
> >> >>>> > > > > >
> >> >>>> > > > > > Xintong Song
> >> >>>> > > > > >
> >> >>>> > > > > >
> >> >>>> > > > > >
> >> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> >> >>>> sewen@apache.org>
> >> >>>> > > > wrote:
> >> >>>> > > > > >
> >> >>>> > > > > > > No need to revert it now - I am not saying it should
> not
> >> go
> >> >>>> into
> >> >>>> > > > > 1.10.1,
> >> >>>> > > > > > I
> >> >>>> > > > > > > am just saying this is not clear to me yet.
> >> >>>> > > > > > >
> >> >>>> > > > > > > We have to trade off the fact that we may break some
> >> >>>> deployments
> >> >>>> > > with
> >> >>>> > > > > the
> >> >>>> > > > > > > fact that we will "safe" a lot of new deployments.
> >> >>>> > > > > > > I simply lack the data points / insight at the moment
> to
> >> >>>> > understand
> >> >>>> > > > how
> >> >>>> > > > > > > significant both cases are, meaning how many users
> would
> >> be
> >> >>>> > > affected
> >> >>>> > > > > and
> >> >>>> > > > > > > how badly.
> >> >>>> > > > > > >
> >> >>>> > > > > > > Independent of that, improving the error message is
> >> always
> >> >>>> > helpful.
> >> >>>> > > > > > >
> >> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> >> >>>> > > > > > > azagrebin.apache@gmail.com>
> >> >>>> > > > > > > wrote:
> >> >>>> > > > > > >
> >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
> >> users
> >> >>>> expect
> >> >>>> > > to
> >> >>>> > > > > > > upgrade
> >> >>>> > > > > > > > > that without config adjustments. That might not be
> >> >>>> possible
> >> >>>> > > with
> >> >>>> > > > > that
> >> >>>> > > > > > > > > change.
> >> >>>> > > > > > > >
> >> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only
> >> try to
> >> >>>> > > improve
> >> >>>> > > > > > error
> >> >>>> > > > > > > > message and docs.
> >> >>>> > > > > > > >
> >> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
> >> >>>> sewen@apache.org>
> >> >>>> > > wrote:
> >> >>>> > > > > > > > >
> >> >>>> > > > > > > > > @Andrey about the increase in metaspace size
> >> >>>> > > > > > > > > - I have no concerns for 1.11.0.
> >> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
> >> users
> >> >>>> expect
> >> >>>> > > to
> >> >>>> > > > > > > upgrade
> >> >>>> > > > > > > > > that without config adjustments. That might not be
> >> >>>> possible
> >> >>>> > > with
> >> >>>> > > > > that
> >> >>>> > > > > > > > > change.
> >> >>>> > > > > > > > >
> >> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> >> >>>> > > > > > > > azagrebin.apache@gmail.com>
> >> >>>> > > > > > > > > wrote:
> >> >>>> > > > > > > > >
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace
> OOM
> >> >>>> error
> >> >>>> > on
> >> >>>> > > > > > repeated
> >> >>>> > > > > > > > >> job”
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> My understanding that the issue is basically
> covered
> >> >>>> by:
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
> >> >>>> handled as
> >> >>>> > > > Fatal
> >> >>>> > > > > > > Error
> >> >>>> > > > > > > > >> in TaskManager
> >> >>>> > > > > > > > >> no full consensus there but improving error
> >> message
> >> >>>> for
> >> >>>> > > > existing
> >> >>>> > > > > > > task
> >> >>>> > > > > > > > >> thread fatal handling could be done at least
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
> >> >>>> Metaspace to
> >> >>>> > > > > minimise
> >> >>>> > > > > > > its
> >> >>>> > > > > > > > >> OutOfMemoryError
> >> >>>> > > > > > > > >> see further
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport"
> >> from
> >> >>>> > > > dynamically
> >> >>>> > > > > > > > loaded
> >> >>>> > > > > > > > >> AWS connectors
> >> >>>> > > > > > > > >> not sure whether this is a blocker but looks
> close
> >> to
> >> >>>> be
> >> >>>> > > > resolved
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> >> >>>> > Metaspace"
> >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> bugfix
> >> >>>> release?
> >> >>>> > > It
> >> >>>> > > > > > > changes
> >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> >> memory
> >> >>>> on
> >> >>>> > > > existing
> >> >>>> > > > > > > setups
> >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> My understanding was that increasing to 256m
> >> resolved
> >> >>>> the
> >> >>>> > > > reported
> >> >>>> > > > > > > > problems
> >> >>>> > > > > > > > >> and we decided to make the change so I have merged
> >> it
> >> >>>> today
> >> >>>> > as
> >> >>>> > > > > there
> >> >>>> > > > > > > > were
> >> >>>> > > > > > > > >> no more concerns.
> >> >>>> > > > > > > > >> If there are concerns I can revert it.
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> On the other hand, I think improving the message
> >> error
> >> >>>> with
> >> >>>> > > > > > reference
> >> >>>> > > > > > > to
> >> >>>> > > > > > > > >> the metaspace option should help the most
> >> >>>> > > > > > > > >> because user would not have to read all docs to
> fix
> >> it
> >> >>>> > > > > > > > >> then maybe this change is not even needed.
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >> Best,
> >> >>>> > > > > > > > >> Andrey
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> >> >>>> sewen@apache.org>
> >> >>>> > > > wrote:
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace
> OOM
> >> >>>> error
> >> >>>> > on
> >> >>>> > > > > > repeated
> >> >>>> > > > > > > > >> job"
> >> >>>> > > > > > > > >>> - I don't think we have consensus on the exact
> >> >>>> solution,
> >> >>>> > yet,
> >> >>>> > > > and
> >> >>>> > > > > > > some
> >> >>>> > > > > > > > >> of
> >> >>>> > > > > > > > >>> the changes might also have side effects that are
> >> >>>> hard to
> >> >>>> > > > > predict,
> >> >>>> > > > > > > so I
> >> >>>> > > > > > > > >> am
> >> >>>> > > > > > > > >>> not sure we should rush this in.
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> >> >>>> > Metaspace"
> >> >>>> > > > > > > > >>> - Have we consensus that this is okay for a
> bugfix
> >> >>>> release?
> >> >>>> > > It
> >> >>>> > > > > > > changes
> >> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
> >> memory
> >> >>>> on
> >> >>>> > > > existing
> >> >>>> > > > > > > setups
> >> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>> We may need to unblock the release form these two
> >> >>>> issues
> >> >>>> > and
> >> >>>> > > > > think
> >> >>>> > > > > > > > about
> >> >>>> > > > > > > > >>> having 1.10.2 in the near future.
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> >> >>>> carp84@gmail.com>
> >> >>>> > > > wrote:
> >> >>>> > > > > > > > >>>
> >> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye
> on
> >> >>>> these
> >> >>>> > two.
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>>> Best Regards,
> >> >>>> > > > > > > > >>>> Yu
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> >> >>>> imjark@gmail.com>
> >> >>>> > > > wrote:
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
> >> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>> From the Table SQL module, I think we should
> also
> >> >>>> try to
> >> >>>> > > get
> >> >>>> > > > in
> >> >>>> > > > > > the
> >> >>>> > > > > > > > >>>>> following issues:
> >> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override
> flink-conf
> >> >>>> > > parameters
> >> >>>> > > > > from
> >> >>>> > > > > > > SQL
> >> >>>> > > > > > > > >> CLI
> >> >>>> > > > > > > > >>>>> environment
> >> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
> >> >>>> watermark
> >> >>>> > > > interval,
> >> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> >> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any
> >> uncatched
> >> >>>> > > > exception
> >> >>>> > > > > is
> >> >>>> > > > > > > > >> thrown
> >> >>>> > > > > > > > >>>>> this will improve much experience when using
> SQL
> >> CLI
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>> Best,
> >> >>>> > > > > > > > >>>>> Jark
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> >> >>>> carp84@gmail.com>
> >> >>>> > > > wrote:
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added
> >> >>>> 1.10.1 into
> >> >>>> > > > > > > FLINK-16225
> >> >>>> > > > > > > > >>>> fix
> >> >>>> > > > > > > > >>>>>> versions and promoted its priority to
> Critical.
> >> >>>> Will
> >> >>>> > also
> >> >>>> > > > > watch
> >> >>>> > > > > > > the
> >> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>> Best Regards,
> >> >>>> > > > > > > > >>>>>> Yu
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin
> <
> >> >>>> > > > > > > azagrebin@apache.org
> >> >>>> > > > > > > > >
> >> >>>> > > > > > > > >>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>>> Hi Yu,
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
> >> >>>> discussion!
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> Apart from
> >> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
> >> >>>> Metaspace
> >> >>>> > to
> >> >>>> > > > > > > minimise
> >> >>>> > > > > > > > >>>> its
> >> >>>> > > > > > > > >>>>>>> OutOfMemoryError
> >> >>>> > > > > > > > >>>>>>> which should be merged soon
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> I think we should also try to get in the
> >> following
> >> >>>> > > issues:
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory
> should
> >> be
> >> >>>> > handled
> >> >>>> > > > as
> >> >>>> > > > > > > Fatal
> >> >>>> > > > > > > > >>>>> Error
> >> >>>> > > > > > > > >>>>>> in
> >> >>>> > > > > > > > >>>>>>> TaskManager
> >> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even
> >> in a
> >> >>>> > better
> >> >>>> > > > way
> >> >>>> > > > > > > > because
> >> >>>> > > > > > > > >>>>> OOM
> >> >>>> > > > > > > > >>>>>>> failure should point users to the docs
> >> immediately
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader
> to
> >> >>>> lifetime
> >> >>>> > > of
> >> >>>> > > > a
> >> >>>> > > > > > slot
> >> >>>> > > > > > > > >>>>>>> This should give a better protection against
> >> class
> >> >>>> > > loading
> >> >>>> > > > > > leaks
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
> >> >>>> submitting
> >> >>>> > > > batch
> >> >>>> > > > > > job
> >> >>>> > > > > > > > >>>>>> (instead
> >> >>>> > > > > > > > >>>>>>> of AskTimeoutException)
> >> >>>> > > > > > > > >>>>>>> This problem has recently happened for
> multiple
> >> >>>> users
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>> Andrey
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> >> >>>> > > > > > > > jingsonglee0@gmail.com>
> >> >>>> > > > > > > > >>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the
> >> >>>> 1.10.1
> >> >>>> > > > release.
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users are
> >> looking
> >> >>>> > > forward
> >> >>>> > > > to
> >> >>>> > > > > > > them.
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>>> Jingsong Lee
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> >> >>>> > > > > > karmagyz@gmail.com>
> >> >>>> > > > > > > > >>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>>>> Yangze Guo
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong
> Song
> >> <
> >> >>>> > > > > > > > >>>>> tonysong820@gmail.com>
> >> >>>> > > > > > > > >>>>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>> Yu,
> >> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
> >> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get
> >> some
> >> >>>> > inputs
> >> >>>> > > > for
> >> >>>> > > > > > > > >>>>>>> prioritizing
> >> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good
> to
> >> >>>> me.
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>> Thank you~
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>> Xintong Song
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> >> >>>> > > > carp84@gmail.com>
> >> >>>> > > > > > > > >>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
> >> >>>> release
> >> >>>> > of
> >> >>>> > > a
> >> >>>> > > > > > major
> >> >>>> > > > > > > > >>>>>>> version
> >> >>>> > > > > > > > >>>>>>>>> will
> >> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
> >> >>>> depending on
> >> >>>> > > the
> >> >>>> > > > > > speed
> >> >>>> > > > > > > > >>>> of
> >> >>>> > > > > > > > >>>>>>>> blocker
> >> >>>> > > > > > > > >>>>>>>>>>> issue resolving:
> >> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th
> >> [1],
> >> >>>> > released
> >> >>>> > > > on
> >> >>>> > > > > > Jul
> >> >>>> > > > > > > > >>>>> 3rd
> >> >>>> > > > > > > > >>>>>>> [2]
> >> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd
> >> [3],
> >> >>>> > released
> >> >>>> > > > on
> >> >>>> > > > > > Oct
> >> >>>> > > > > > > > >>>>> 19th
> >> >>>> > > > > > > > >>>>>>> [4]
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of
> >> course,
> >> >>>> but
> >> >>>> > > could
> >> >>>> > > > > use
> >> >>>> > > > > > > > >>>> it
> >> >>>> > > > > > > > >>>>>> as a
> >> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me
> >> >>>> know if
> >> >>>> > any
> >> >>>> > > > > > > > >>>> concerns
> >> >>>> > > > > > > > >>>>>>>> Xintong.
> >> >>>> > > > > > > > >>>>>>>>>>> Thanks.
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> Best Regards,
> >> >>>> > > > > > > > >>>>>>>>>>> Yu
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> [1]
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > >
> >> >>>> > > > > > >
> >> >>>> > > > > >
> >> >>>> > > > >
> >> >>>> > > >
> >> >>>> > >
> >> >>>> >
> >> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >> >>>> > > > > > > > >>>>>>>>>>> [2]
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > >
> >> >>>> > > > > > >
> >> >>>> > > > > >
> >> >>>> > > > >
> >> >>>> > > >
> >> >>>> > >
> >> >>>> >
> >> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >> >>>> > > > > > > > >>>>>>>>>>> [3]
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > >
> >> >>>> > > > > > >
> >> >>>> > > > > >
> >> >>>> > > > >
> >> >>>> > > >
> >> >>>> > >
> >> >>>> >
> >> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >> >>>> > > > > > > > >>>>>>>>>>> [4]
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > >
> >> >>>> > > > > > >
> >> >>>> > > > > >
> >> >>>> > > > >
> >> >>>> > > >
> >> >>>> > >
> >> >>>> >
> >> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong
> Song
> >> <
> >> >>>> > > > > > > > >>>>>> tonysong820@gmail.com>
> >> >>>> > > > > > > > >>>>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and
> >> volunteering
> >> >>>> to be
> >> >>>> > > the
> >> >>>> > > > > > > > >>>>> release
> >> >>>> > > > > > > > >>>>>>>>> manager.
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time
> plan
> >> for
> >> >>>> > > 1.10.1?
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> Thank you~
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM
> Zhijiang
> >> >>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid>
> >> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> >> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > >
> >> >>>> >
> ------------------------------------------------------------------
> >> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink
> >> 1.10.1
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes
> will
> >> >>>> also
> >> >>>> > keep
> >> >>>> > > > an
> >> >>>> > > > > > > > >>>> eye
> >> >>>> > > > > > > > >>>>> on
> >> >>>> > > > > > > > >>>>>>>> these
> >> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
> >> >>>> > > > > > > > >>>>>>>>>>>>> Yu
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
> >> Cheng <
> >> >>>> > > > > > > > >>>>> hequn@apache.org>
> >> >>>> > > > > > > > >>>>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the
> discussion
> >> and
> >> >>>> > > > volunteer
> >> >>>> > > > > > > > >>>> as
> >> >>>> > > > > > > > >>>>>> the
> >> >>>> > > > > > > > >>>>>>>>> release
> >> >>>> > > > > > > > >>>>>>>>>>>>>> manager!
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1]
> >> >>>> which are
> >> >>>> > > > > marked
> >> >>>> > > > > > > > >>>>> as
> >> >>>> > > > > > > > >>>>>> a
> >> >>>> > > > > > > > >>>>>>>>> blocker:
> >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright
> year
> >> in
> >> >>>> > NOTICE
> >> >>>> > > > > files
> >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem
> with
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> >> >>>> and
> >> >>>> > > usrlib
> >> >>>> > > > > > > > >>>>>> directory
> >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> >> >>>> > > > > > > > >>>>> ClassNotFoundException
> >> >>>> > > > > > > > >>>>>>> when
> >> >>>> > > > > > > > >>>>>>>>> use
> >> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting
> >> when
> >> >>>> > > > submitting
> >> >>>> > > > > > > > >>>>> batch
> >> >>>> > > > > > > > >>>>>>> job
> >> >>>> > > > > > > > >>>>>>>>>>>> (instead
> >> >>>> > > > > > > > >>>>>>>>>>>>> of
> >> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in
> >> >>>> 1.10.1.
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> [1]
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > >
> >> >>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li
> <
> >> >>>> > > > > > > > >>>> carp84@gmail.com>
> >> >>>> > > > > > > > >>>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
> >> >>>> Thanks a
> >> >>>> > > lot!
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24,
> jincheng
> >> >>>> sun <
> >> >>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion
> >> Yu. I
> >> >>>> would
> >> >>>> > > > like
> >> >>>> > > > > > > > >>>>> to
> >> >>>> > > > > > > > >>>>>>> give
> >> >>>> > > > > > > > >>>>>>>>> you a
> >> >>>> > > > > > > > >>>>>>>>>>>>> hand
> >> >>>> > > > > > > > >>>>>>>>>>>>>> at
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
> >> >>>> finished.(If you
> >> >>>> > > > > > > > >>>> need)
> >> >>>> > > > > > > > >>>>>> :)
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
> >> 于2020年3月10日周二
> >> >>>> > > 下午5:49写道：
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since
> we
> >> >>>> > released
> >> >>>> > > > > > > > >>>>> Flink
> >> >>>> > > > > > > > >>>>>>>>> 1.10.0. We
> >> >>>> > > > > > > > >>>>>>>>>>>>>> already
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
> >> >>>> improvements/bugs
> >> >>>> > in
> >> >>>> > > > > > > > >>>> the
> >> >>>> > > > > > > > >>>>>>>>>>> release-1.10
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> branch,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1
> >> >>>> release
> >> >>>> > > cycle.
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the
> >> >>>> license and
> >> >>>> > > > > > > > >>>> notice
> >> >>>> > > > > > > > >>>>>>> file
> >> >>>> > > > > > > > >>>>>>>> in
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource
> >> leak
> >> >>>> in
> >> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero
> should
> >> be
> >> >>>> known
> >> >>>> > > in
> >> >>>> > > > > > > > >>>>>>>>> HiveCatalog
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
> >> >>>> ArrayIndexOufOBoundsException in
> >> >>>> > > > > > > > >>>>>>>> TypeExtractor
> >> >>>> > > > > > > > >>>>>>>>>>> when
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is
> >> >>>> failed if
> >> >>>> > > > > > > > >>>> running
> >> >>>> > > > > > > > >>>>>> in
> >> >>>> > > > > > > > >>>>>>>>> blink
> >> >>>> > > > > > > > >>>>>>>>>>>>> planner
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location
> constraints
> >> >>>> are not
> >> >>>> > > > > > > > >>>> reset
> >> >>>> > > > > > > > >>>>>> on
> >> >>>> > > > > > > > >>>>>>>> task
> >> >>>> > > > > > > > >>>>>>>>>>>>> recovery
> >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> >> function
> >> >>>> using
> >> >>>> > > sql
> >> >>>> > > > > > > > >>>>>>> casuing
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation
> >> >>>> failed
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following
> >> >>>> issues
> >> >>>> > > should
> >> >>>> > > > > > > > >>>> be
> >> >>>> > > > > > > > >>>>>>>> merged
> >> >>>> > > > > > > > >>>>>>>>>>> before
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace
> OOM
> >> >>>> issue):
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
> >> >>>> Metaspace
> >> >>>> > OOM
> >> >>>> > > > > > > > >>>>> error
> >> >>>> > > > > > > > >>>>>> on
> >> >>>> > > > > > > > >>>>>>>>>>> repeated
> >> >>>> > > > > > > > >>>>>>>>>>>>> job
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default
> value
> >> >>>> for JVM
> >> >>>> > > > > > > > >>>>>> Metaspace
> >> >>>> > > > > > > > >>>>>>> to
> >> >>>> > > > > > > > >>>>>>>>>>>> minimise
> >> >>>> > > > > > > > >>>>>>>>>>>>>> its
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner
> produces
> >> >>>> wrong
> >> >>>> > > > > > > > >>>>> aggregate
> >> >>>> > > > > > > > >>>>>>>>> results
> >> >>>> > > > > > > > >>>>>>>>>>> with
> >> >>>> > > > > > > > >>>>>>>>>>>>>> state
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
> >> >>>> extract
> >> >>>> > > > > > > > >>>> correct
> >> >>>> > > > > > > > >>>>>>>> unique
> >> >>>> > > > > > > > >>>>>>>>> key
> >> >>>> > > > > > > > >>>>>>>>>>>> for
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release
> >> >>>> manager and
> >> >>>> > > kick
> >> >>>> > > > > > > > >>>>> off
> >> >>>> > > > > > > > >>>>>>> the
> >> >>>> > > > > > > > >>>>>>>>>>> release
> >> >>>> > > > > > > > >>>>>>>>>>>>>>> process
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged.
> What
> >> do
> >> >>>> you
> >> >>>> > > > > > > > >>>> think?
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or
> missing
> >> >>>> blocker
> >> >>>> > > > > > > > >>>> issues
> >> >>>> > > > > > > > >>>>>>> need
> >> >>>> > > > > > > > >>>>>>>>> to be
> >> >>>> > > > > > > > >>>>>>>>>>>>> fixed
> >> >>>> > > > > > > > >>>>>>>>>>>>>> in
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>> --
> >> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
> >> >>>> > > > > > > > >>>>>>>>
> >> >>>> > > > > > > > >>>>>>>
> >> >>>> > > > > > > > >>>>>>
> >> >>>> > > > > > > > >>>>>
> >> >>>> > > > > > > > >>>>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > > >>
> >> >>>> > > > > > > >
> >> >>>> > > > > > > >
> >> >>>> > > > > > >
> >> >>>> > > > > >
> >> >>>> > > > >
> >> >>>> > > >
> >> >>>> > >
> >> >>>> >
> >> >>>>
> >> >>>
> >>
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks Weike and all others for the efforts!
Here comes the latest status, we are in good shape and plan to produce RC1
next week.
* Blockers (1 left)
- [Closed] FLINK-16018 Improve error reporting when submitting batch job
(instead of AskTimeoutException)
- [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
job submission
- [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
use flink-sql-connector-elasticsearch7
- [Closed] FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- [Closed] FLINK-16406 Increase default value for JVM Metaspace to
minimise its OutOfMemoryError
- [Closed] FLINK-16454 Update the copyright year in NOTICE files
- [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before client
can retrieve JobResult
- [Closed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
throws UnsupportedOperationException
- [Closed] FLINK-16626 Exception encountered when cancelling a job in
yarn per-job mode
- [Fix for 1.10.1 is Done] FLINK-17093 Python UDF doesn't work when the
input column is of composite type
- [PR reviewed] FLINK-16576 State inconsistency on restore with memory
state backends
* Critical (1 left)
- [Closed] FLINK-16047 Blink planner produces wrong aggregate results
with state clean up
- [Closed] FLINK-16070 Blink planner can not extract correct unique key
for UpsertStreamTableSink
- [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be
handled as Fatal Error in TaskManager
- [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
- [May Postpone] FLINK-16408 Bind user code class loader to lifetime of a
slot
Please let me know if any concerns/comments. Thanks.
Best Regards,
Yu
On Fri, 3 Apr 2020 at 21:35, DONG, Weike <ky...@connect.hku.hk> wrote:
> Hi Yu,
>
> Thanks for your updates. I am still working on the fix for FLINK-16626 and
> it is expected to be completed by this Sunday after thorough testing.
>
> Sincerely,
> Weike
>
> On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
>
>> Updates for 1.10.1 watched issues (we are in good progress and almost
>> there
>> to produce the first RC, thanks all for the efforts):
>>
>> * Blockers (3 left)
>> - [Closed] FLINK-16018 Improve error reporting when submitting batch job
>> (instead of AskTimeoutException)
>> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>> repeated
>> job submission
>> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
>> use flink-sql-connector-elasticsearch7
>> - [Closed] FLINK-16262 Class loader problem with
>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> minimise its OutOfMemoryError
>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>> - [PR reviewed] FLINK-16576 State inconsistency on restore with memory
>> state backends
>> - [Under Discussion] FLINK-16626 Exception encountered when cancelling a
>> job in yarn per-job mode
>> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before
>> client
>> can retrieve JobResult
>> - [PR reviewed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
>> throws UnsupportedOperationException
>>
>> * Critical (1 left)
>> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
>> with state clean up
>> - [Closed] FLINK-16070 Blink planner can not extract correct unique key
>> for UpsertStreamTableSink
>> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be
>> handled as Fatal Error in TaskManager
>> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime of
>> a
>> slot
>>
>> Please let me know if you find any missing ones, thanks.
>>
>> Best Regards,
>> Yu
>>
>>
>> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
>>
>> > Here comes the latest status of issues in 1.10.1 watch list:
>> >
>> > * Blockers (4 left)
>> > - [Under Discussion] FLINK-16018 Improve error reporting when
>> submitting
>> > batch job (instead of AskTimeoutException)
>> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
>> > repeated job submission
>> > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
>> when
>> > use flink-sql-connector-elasticsearch7
>> > - [PR Approved] FLINK-16262 Class loader problem with
>> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> > minimise its OutOfMemoryError
>> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>> > - [In Progress] FLINK-16576 State inconsistency on restore with memory
>> > state backends
>> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down
>> MiniCluster
>> > before client can retrieve JobResult
>> >
>> > * Critical (2 left)
>> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
>> > with state clean up
>> > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>> > unique key for UpsertStreamTableSink
>> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should be
>> > handled as Fatal Error in TaskManager
>> > - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
>> >
>> > There's a new blocker added. Please let me know if you find any other
>> > issues should be put into the watch list. Thanks.
>> >
>> > Best Regards,
>> > Yu
>> >
>> >
>> > On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
>> >
>> >> Hi All,
>> >>
>> >> Here is the status update of issues in 1.10.1 watch list:
>> >>
>> >> * Blockers (7)
>> >>
>> >> - [Under Discussion] FLINK-16018 Improve error reporting when
>> >> submitting batch job (instead of AskTimeoutException)
>> >>
>> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
>> error
>> >> on repeated job submission
>> >>
>> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> >> ClassNotFoundException when use flink-sql-connector-elasticsearch7
>> >>
>> >> - [PR Submitted] FLINK-16262 Class loader problem with
>> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> >>
>> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> >> minimise its OutOfMemoryError
>> >>
>> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>> >>
>> >> - [Open] FLINK-16576 State inconsistency on restore with memory state
>> >> backends
>> >>
>> >>
>> >> * Critical (4)
>> >>
>> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
>> >> with state clean up
>> >>
>> >> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>> >> unique key for UpsertStreamTableSink
>> >>
>> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
>> >> handled as Fatal Error in TaskManager
>> >>
>> >> - [Open] FLINK-16408 Bind user code class loader to lifetime of a
>> slot
>> >>
>> >> Please let me know if you see any new blockers to add to the list.
>> Thanks.
>> >>
>> >> Best Regards,
>> >> Yu
>> >>
>> >>
>> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
>> >>
>> >>> Thanks for the updates Till!
>> >>>
>> >>> For FLINK-16018, maybe we could create two sub-tasks for easy and
>> >>> complete fix separately, and only include the easy one in 1.10.1? Or
>> please
>> >>> just feel free to postpone the whole task to 1.10.2 if "all or
>> nothing"
>> >>> policy is preferred (smile). Thanks.
>> >>>
>> >>> Best Regards,
>> >>> Yu
>> >>>
>> >>>
>> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org>
>> >>> wrote:
>> >>>
>> >>>> +1 for a soonish bug fix release. Thanks for volunteering as our
>> release
>> >>>> manager Yu.
>> >>>>
>> >>>> I think we can soon merge the increase of metaspace size and
>> improving
>> >>>> the
>> >>>> error message. The assumption is that we currently don't have too
>> many
>> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of course,
>> the
>> >>>> sooner we release the bug fix release, the fewer deployments will be
>> >>>> affected by this change.
>> >>>>
>> >>>> For FLINK-16018, I think there would be an easy solution which we
>> could
>> >>>> include in the bug fix release. The proper fix will most likely take
>> a
>> >>>> bit
>> >>>> longer, though.
>> >>>>
>> >>>> Cheers,
>> >>>> Till
>> >>>>
>> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <
>> azagrebin@apache.org>
>> >>>> wrote:
>> >>>>
>> >>>> > > @Andrey and @Xintong - could we have a quick poll on the user
>> >>>> mailing
>> >>>> > list
>> >>>> > > about increasing the metaspace size in Flink 1.10.1? Specifically
>> >>>> asking
>> >>>> > > for who has very small TM setups?
>> >>>> >
>> >>>> > There has been a survey about this topic since 10 days:
>> >>>> >
>> >>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
>> >>>> > I can ask there about the small TM setups.
>> >>>> >
>> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
>> >>>> >
>> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on
>> >>>> restore
>> >>>> > with
>> >>>> > > memory state backends
>> >>>> > >
>> >>>> > > Let me recompile the watching list with recent feedbacks.
>> There're
>> >>>> > totally
>> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of
>> which 14
>> >>>> > > already resolved and 31 left, and the below ones are watched
>> >>>> (meaning
>> >>>> > that
>> >>>> > > once the below ones got fixed, we will kick of the releasing with
>> >>>> left
>> >>>> > ones
>> >>>> > > postponed to 1.10.2 unless objections):
>> >>>> > >
>> >>>> > > * Blockers (7)
>> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
>> >>>> > submitting
>> >>>> > > batch job (instead of AskTimeoutException)
>> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
>> OOM
>> >>>> error
>> >>>> > > on repeated job submission
>> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> >>>> > ClassNotFoundException
>> >>>> > > when use flink-sql-connector-elasticsearch7
>> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
>> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM
>> Metaspace to
>> >>>> > > minimise its OutOfMemoryError
>> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
>> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with memory
>> >>>> state
>> >>>> > > backends
>> >>>> > >
>> >>>> > > * Critical (4)
>> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate
>> >>>> results
>> >>>> > with
>> >>>> > > state clean up
>> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract
>> correct
>> >>>> > unique
>> >>>> > > key for UpsertStreamTableSink
>> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory
>> should be
>> >>>> > > handled as Fatal Error in TaskManager
>> >>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime
>> of a
>> >>>> slot
>> >>>> > >
>> >>>> > > Please raise your hand if you find any other issues should be put
>> >>>> into
>> >>>> > this
>> >>>> > > list, thanks.
>> >>>> > >
>> >>>> > > Please also note that 1.10.2 version is already created (thanks
>> for
>> >>>> the
>> >>>> > > help @jincheng), and feel free to adjust/assign fix version to it
>> >>>> when
>> >>>> > > necessary.
>> >>>> > >
>> >>>> > > Best Regards,
>> >>>> > > Yu
>> >>>> > >
>> >>>> > >
>> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org>
>> >>>> wrote:
>> >>>> > >
>> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the user
>> >>>> mailing
>> >>>> > > list
>> >>>> > > > about increasing the metaspace size in Flink 1.10.1?
>> Specifically
>> >>>> > asking
>> >>>> > > > for who has very small TM setups?
>> >>>> > > >
>> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com>
>> wrote:
>> >>>> > > >
>> >>>> > > > > Thanks for the reminder Stephan and the inputs/discussion
>> >>>> Andrey and
>> >>>> > > > > Xintong. I totally agree that we should try best not to
>> >>>> introduce new
>> >>>> > > > > issues when resolving the existing one. Will watch closely
>> >>>> about the
>> >>>> > > > > discussion and conclusion around metaspace configuration.
>> >>>> > > > >
>> >>>> > > > > From the point as a developer instead of RM, I'm optimistic
>> that
>> >>>> > > > reserving
>> >>>> > > > > 160MB more memory to JVM metaspace by default won't cause a
>> big
>> >>>> > > problem,
>> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB
>> memory
>> >>>> per TM
>> >>>> > > is
>> >>>> > > > > the common case.
>> >>>> > > > >
>> >>>> > > > > Best Regards,
>> >>>> > > > > Yu
>> >>>> > > > >
>> >>>> > > > >
>> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
>> >>>> tonysong820@gmail.com>
>> >>>> > > > wrote:
>> >>>> > > > >
>> >>>> > > > > > I think in FLINK-16406 we have not only increased the
>> default
>> >>>> value
>> >>>> > > for
>> >>>> > > > > > metaspace, but also increased the 'process.size' in default
>> >>>> > > > > > 'flink-conf.yaml'.
>> >>>> > > > > >
>> >>>> > > > > > That means:
>> >>>> > > > > >
>> >>>> > > > > > - For people using the default 'flink-conf.yaml', it's
>> not
>> >>>> > really
>> >>>> > > > > taking
>> >>>> > > > > > memory from heap / managed for metaspace.
>> >>>> > > > > > - For people using custom configuration files:
>> >>>> > > > > > - If they are specifying 'flink.size', then metaspace
>> >>>> does
>> >>>> > not
>> >>>> > > > > really
>> >>>> > > > > > affect the flink memory internal components (heap,
>> >>>> managed,
>> >>>> > > > etc.).
>> >>>> > > > > > - If they are specifying 'process.size' (most likely
>> >>>> active
>> >>>> > > > > setups),
>> >>>> > > > > > then they will have less heap / managed memory, or
>> >>>> worse,
>> >>>> > their
>> >>>> > > > > > configuration may not work without adjustment.
>> >>>> > > > > > - For a TM with large process memory (e.g., 4 GB
>> or
>> >>>> more),
>> >>>> > > it
>> >>>> > > > is
>> >>>> > > > > > probably ok that the metaspace takes 160 MB more
>> >>>> memory.
>> >>>> > > > > > - For a TM with smaller total memory (1 or 2 GB)
>> the
>> >>>> > > metaspace
>> >>>> > > > > > increase is more likely to cause problem.
>> >>>> > > > > >
>> >>>> > > > > > So basically only people have small 'process.size' in
>> custom
>> >>>> config
>> >>>> > > > file
>> >>>> > > > > > are really affected. I'm not sure what is the proportion of
>> >>>> such
>> >>>> > use
>> >>>> > > > > cases
>> >>>> > > > > > though. (From questions asked on the user ML, probably not
>> >>>> much).
>> >>>> > > > > >
>> >>>> > > > > > Thank you~
>> >>>> > > > > >
>> >>>> > > > > > Xintong Song
>> >>>> > > > > >
>> >>>> > > > > >
>> >>>> > > > > >
>> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
>> >>>> sewen@apache.org>
>> >>>> > > > wrote:
>> >>>> > > > > >
>> >>>> > > > > > > No need to revert it now - I am not saying it should not
>> go
>> >>>> into
>> >>>> > > > > 1.10.1,
>> >>>> > > > > > I
>> >>>> > > > > > > am just saying this is not clear to me yet.
>> >>>> > > > > > >
>> >>>> > > > > > > We have to trade off the fact that we may break some
>> >>>> deployments
>> >>>> > > with
>> >>>> > > > > the
>> >>>> > > > > > > fact that we will "safe" a lot of new deployments.
>> >>>> > > > > > > I simply lack the data points / insight at the moment to
>> >>>> > understand
>> >>>> > > > how
>> >>>> > > > > > > significant both cases are, meaning how many users would
>> be
>> >>>> > > affected
>> >>>> > > > > and
>> >>>> > > > > > > how badly.
>> >>>> > > > > > >
>> >>>> > > > > > > Independent of that, improving the error message is
>> always
>> >>>> > helpful.
>> >>>> > > > > > >
>> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>> >>>> > > > > > > azagrebin.apache@gmail.com>
>> >>>> > > > > > > wrote:
>> >>>> > > > > > >
>> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
>> users
>> >>>> expect
>> >>>> > > to
>> >>>> > > > > > > upgrade
>> >>>> > > > > > > > > that without config adjustments. That might not be
>> >>>> possible
>> >>>> > > with
>> >>>> > > > > that
>> >>>> > > > > > > > > change.
>> >>>> > > > > > > >
>> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only
>> try to
>> >>>> > > improve
>> >>>> > > > > > error
>> >>>> > > > > > > > message and docs.
>> >>>> > > > > > > >
>> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
>> >>>> sewen@apache.org>
>> >>>> > > wrote:
>> >>>> > > > > > > > >
>> >>>> > > > > > > > > @Andrey about the increase in metaspace size
>> >>>> > > > > > > > > - I have no concerns for 1.11.0.
>> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because
>> users
>> >>>> expect
>> >>>> > > to
>> >>>> > > > > > > upgrade
>> >>>> > > > > > > > > that without config adjustments. That might not be
>> >>>> possible
>> >>>> > > with
>> >>>> > > > > that
>> >>>> > > > > > > > > change.
>> >>>> > > > > > > > >
>> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
>> >>>> > > > > > > > azagrebin.apache@gmail.com>
>> >>>> > > > > > > > > wrote:
>> >>>> > > > > > > > >
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>> >>>> error
>> >>>> > on
>> >>>> > > > > > repeated
>> >>>> > > > > > > > >> job”
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> My understanding that the issue is basically covered
>> >>>> by:
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
>> >>>> handled as
>> >>>> > > > Fatal
>> >>>> > > > > > > Error
>> >>>> > > > > > > > >> in TaskManager
>> >>>> > > > > > > > >> no full consensus there but improving error
>> message
>> >>>> for
>> >>>> > > > existing
>> >>>> > > > > > > task
>> >>>> > > > > > > > >> thread fatal handling could be done at least
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
>> >>>> Metaspace to
>> >>>> > > > > minimise
>> >>>> > > > > > > its
>> >>>> > > > > > > > >> OutOfMemoryError
>> >>>> > > > > > > > >> see further
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport"
>> from
>> >>>> > > > dynamically
>> >>>> > > > > > > > loaded
>> >>>> > > > > > > > >> AWS connectors
>> >>>> > > > > > > > >> not sure whether this is a blocker but looks close
>> to
>> >>>> be
>> >>>> > > > resolved
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>> >>>> > Metaspace"
>> >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>> >>>> release?
>> >>>> > > It
>> >>>> > > > > > > changes
>> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
>> memory
>> >>>> on
>> >>>> > > > existing
>> >>>> > > > > > > setups
>> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> My understanding was that increasing to 256m
>> resolved
>> >>>> the
>> >>>> > > > reported
>> >>>> > > > > > > > problems
>> >>>> > > > > > > > >> and we decided to make the change so I have merged
>> it
>> >>>> today
>> >>>> > as
>> >>>> > > > > there
>> >>>> > > > > > > > were
>> >>>> > > > > > > > >> no more concerns.
>> >>>> > > > > > > > >> If there are concerns I can revert it.
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> On the other hand, I think improving the message
>> error
>> >>>> with
>> >>>> > > > > > reference
>> >>>> > > > > > > to
>> >>>> > > > > > > > >> the metaspace option should help the most
>> >>>> > > > > > > > >> because user would not have to read all docs to fix
>> it
>> >>>> > > > > > > > >> then maybe this change is not even needed.
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >> Best,
>> >>>> > > > > > > > >> Andrey
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>> >>>> sewen@apache.org>
>> >>>> > > > wrote:
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>> >>>> error
>> >>>> > on
>> >>>> > > > > > repeated
>> >>>> > > > > > > > >> job"
>> >>>> > > > > > > > >>> - I don't think we have consensus on the exact
>> >>>> solution,
>> >>>> > yet,
>> >>>> > > > and
>> >>>> > > > > > > some
>> >>>> > > > > > > > >> of
>> >>>> > > > > > > > >>> the changes might also have side effects that are
>> >>>> hard to
>> >>>> > > > > predict,
>> >>>> > > > > > > so I
>> >>>> > > > > > > > >> am
>> >>>> > > > > > > > >>> not sure we should rush this in.
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>> >>>> > Metaspace"
>> >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>> >>>> release?
>> >>>> > > It
>> >>>> > > > > > > changes
>> >>>> > > > > > > > >>> setups, takes away memory from heap / managed
>> memory
>> >>>> on
>> >>>> > > > existing
>> >>>> > > > > > > setups
>> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>> We may need to unblock the release form these two
>> >>>> issues
>> >>>> > and
>> >>>> > > > > think
>> >>>> > > > > > > > about
>> >>>> > > > > > > > >>> having 1.10.2 in the near future.
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
>> >>>> carp84@gmail.com>
>> >>>> > > > wrote:
>> >>>> > > > > > > > >>>
>> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on
>> >>>> these
>> >>>> > two.
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>>> Best Regards,
>> >>>> > > > > > > > >>>> Yu
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>> >>>> imjark@gmail.com>
>> >>>> > > > wrote:
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
>> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>> From the Table SQL module, I think we should also
>> >>>> try to
>> >>>> > > get
>> >>>> > > > in
>> >>>> > > > > > the
>> >>>> > > > > > > > >>>>> following issues:
>> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
>> >>>> > > parameters
>> >>>> > > > > from
>> >>>> > > > > > > SQL
>> >>>> > > > > > > > >> CLI
>> >>>> > > > > > > > >>>>> environment
>> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
>> >>>> watermark
>> >>>> > > > interval,
>> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
>> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any
>> uncatched
>> >>>> > > > exception
>> >>>> > > > > is
>> >>>> > > > > > > > >> thrown
>> >>>> > > > > > > > >>>>> this will improve much experience when using SQL
>> CLI
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>> Best,
>> >>>> > > > > > > > >>>>> Jark
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
>> >>>> carp84@gmail.com>
>> >>>> > > > wrote:
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added
>> >>>> 1.10.1 into
>> >>>> > > > > > > FLINK-16225
>> >>>> > > > > > > > >>>> fix
>> >>>> > > > > > > > >>>>>> versions and promoted its priority to Critical.
>> >>>> Will
>> >>>> > also
>> >>>> > > > > watch
>> >>>> > > > > > > the
>> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>> Best Regards,
>> >>>> > > > > > > > >>>>>> Yu
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
>> >>>> > > > > > > azagrebin@apache.org
>> >>>> > > > > > > > >
>> >>>> > > > > > > > >>>>>> wrote:
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>>> Hi Yu,
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
>> >>>> discussion!
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> Apart from
>> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
>> >>>> Metaspace
>> >>>> > to
>> >>>> > > > > > > minimise
>> >>>> > > > > > > > >>>> its
>> >>>> > > > > > > > >>>>>>> OutOfMemoryError
>> >>>> > > > > > > > >>>>>>> which should be merged soon
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> I think we should also try to get in the
>> following
>> >>>> > > issues:
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
>> be
>> >>>> > handled
>> >>>> > > > as
>> >>>> > > > > > > Fatal
>> >>>> > > > > > > > >>>>> Error
>> >>>> > > > > > > > >>>>>> in
>> >>>> > > > > > > > >>>>>>> TaskManager
>> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even
>> in a
>> >>>> > better
>> >>>> > > > way
>> >>>> > > > > > > > because
>> >>>> > > > > > > > >>>>> OOM
>> >>>> > > > > > > > >>>>>>> failure should point users to the docs
>> immediately
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
>> >>>> lifetime
>> >>>> > > of
>> >>>> > > > a
>> >>>> > > > > > slot
>> >>>> > > > > > > > >>>>>>> This should give a better protection against
>> class
>> >>>> > > loading
>> >>>> > > > > > leaks
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
>> >>>> submitting
>> >>>> > > > batch
>> >>>> > > > > > job
>> >>>> > > > > > > > >>>>>> (instead
>> >>>> > > > > > > > >>>>>>> of AskTimeoutException)
>> >>>> > > > > > > > >>>>>>> This problem has recently happened for multiple
>> >>>> users
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> Best,
>> >>>> > > > > > > > >>>>>>> Andrey
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
>> >>>> > > > > > > > jingsonglee0@gmail.com>
>> >>>> > > > > > > > >>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the
>> >>>> 1.10.1
>> >>>> > > > release.
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users are
>> looking
>> >>>> > > forward
>> >>>> > > > to
>> >>>> > > > > > > them.
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>> Best,
>> >>>> > > > > > > > >>>>>>>> Jingsong Lee
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
>> >>>> > > > > > karmagyz@gmail.com>
>> >>>> > > > > > > > >>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>> Best,
>> >>>> > > > > > > > >>>>>>>>> Yangze Guo
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song
>> <
>> >>>> > > > > > > > >>>>> tonysong820@gmail.com>
>> >>>> > > > > > > > >>>>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>> Yu,
>> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
>> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get
>> some
>> >>>> > inputs
>> >>>> > > > for
>> >>>> > > > > > > > >>>>>>> prioritizing
>> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to
>> >>>> me.
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>> Thank you~
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>> Xintong Song
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
>> >>>> > > > carp84@gmail.com>
>> >>>> > > > > > > > >>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
>> >>>> release
>> >>>> > of
>> >>>> > > a
>> >>>> > > > > > major
>> >>>> > > > > > > > >>>>>>> version
>> >>>> > > > > > > > >>>>>>>>> will
>> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
>> >>>> depending on
>> >>>> > > the
>> >>>> > > > > > speed
>> >>>> > > > > > > > >>>> of
>> >>>> > > > > > > > >>>>>>>> blocker
>> >>>> > > > > > > > >>>>>>>>>>> issue resolving:
>> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th
>> [1],
>> >>>> > released
>> >>>> > > > on
>> >>>> > > > > > Jul
>> >>>> > > > > > > > >>>>> 3rd
>> >>>> > > > > > > > >>>>>>> [2]
>> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd
>> [3],
>> >>>> > released
>> >>>> > > > on
>> >>>> > > > > > Oct
>> >>>> > > > > > > > >>>>> 19th
>> >>>> > > > > > > > >>>>>>> [4]
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of
>> course,
>> >>>> but
>> >>>> > > could
>> >>>> > > > > use
>> >>>> > > > > > > > >>>> it
>> >>>> > > > > > > > >>>>>> as a
>> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me
>> >>>> know if
>> >>>> > any
>> >>>> > > > > > > > >>>> concerns
>> >>>> > > > > > > > >>>>>>>> Xintong.
>> >>>> > > > > > > > >>>>>>>>>>> Thanks.
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> Best Regards,
>> >>>> > > > > > > > >>>>>>>>>>> Yu
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> [1]
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > >
>> >>>> > > > > > >
>> >>>> > > > > >
>> >>>> > > > >
>> >>>> > > >
>> >>>> > >
>> >>>> >
>> >>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>> >>>> > > > > > > > >>>>>>>>>>> [2]
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > >
>> >>>> > > > > > >
>> >>>> > > > > >
>> >>>> > > > >
>> >>>> > > >
>> >>>> > >
>> >>>> >
>> >>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>> >>>> > > > > > > > >>>>>>>>>>> [3]
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > >
>> >>>> > > > > > >
>> >>>> > > > > >
>> >>>> > > > >
>> >>>> > > >
>> >>>> > >
>> >>>> >
>> >>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>> >>>> > > > > > > > >>>>>>>>>>> [4]
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > >
>> >>>> > > > > > >
>> >>>> > > > > >
>> >>>> > > > >
>> >>>> > > >
>> >>>> > >
>> >>>> >
>> >>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song
>> <
>> >>>> > > > > > > > >>>>>> tonysong820@gmail.com>
>> >>>> > > > > > > > >>>>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and
>> volunteering
>> >>>> to be
>> >>>> > > the
>> >>>> > > > > > > > >>>>> release
>> >>>> > > > > > > > >>>>>>>>> manager.
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan
>> for
>> >>>> > > 1.10.1?
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> Thank you~
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>> >>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid>
>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
>> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>> Best,
>> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > >
>> >>>> > ------------------------------------------------------------------
>> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink
>> 1.10.1
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will
>> >>>> also
>> >>>> > keep
>> >>>> > > > an
>> >>>> > > > > > > > >>>> eye
>> >>>> > > > > > > > >>>>> on
>> >>>> > > > > > > > >>>>>>>> these
>> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
>> >>>> > > > > > > > >>>>>>>>>>>>> Yu
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn
>> Cheng <
>> >>>> > > > > > > > >>>>> hequn@apache.org>
>> >>>> > > > > > > > >>>>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion
>> and
>> >>>> > > > volunteer
>> >>>> > > > > > > > >>>> as
>> >>>> > > > > > > > >>>>>> the
>> >>>> > > > > > > > >>>>>>>>> release
>> >>>> > > > > > > > >>>>>>>>>>>>>> manager!
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1]
>> >>>> which are
>> >>>> > > > > marked
>> >>>> > > > > > > > >>>>> as
>> >>>> > > > > > > > >>>>>> a
>> >>>> > > > > > > > >>>>>>>>> blocker:
>> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year
>> in
>> >>>> > NOTICE
>> >>>> > > > > files
>> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>> >>>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
>> >>>> and
>> >>>> > > usrlib
>> >>>> > > > > > > > >>>>>> directory
>> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>> >>>> > > > > > > > >>>>> ClassNotFoundException
>> >>>> > > > > > > > >>>>>>> when
>> >>>> > > > > > > > >>>>>>>>> use
>> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting
>> when
>> >>>> > > > submitting
>> >>>> > > > > > > > >>>>> batch
>> >>>> > > > > > > > >>>>>>> job
>> >>>> > > > > > > > >>>>>>>>>>>> (instead
>> >>>> > > > > > > > >>>>>>>>>>>>> of
>> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in
>> >>>> 1.10.1.
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> Best,
>> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> [1]
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > >
>> >>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>> >>>> > > > > > > > >>>> carp84@gmail.com>
>> >>>> > > > > > > > >>>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
>> >>>> Thanks a
>> >>>> > > lot!
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
>> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng
>> >>>> sun <
>> >>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion
>> Yu. I
>> >>>> would
>> >>>> > > > like
>> >>>> > > > > > > > >>>>> to
>> >>>> > > > > > > > >>>>>>> give
>> >>>> > > > > > > > >>>>>>>>> you a
>> >>>> > > > > > > > >>>>>>>>>>>>> hand
>> >>>> > > > > > > > >>>>>>>>>>>>>> at
>> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
>> >>>> finished.(If you
>> >>>> > > > > > > > >>>> need)
>> >>>> > > > > > > > >>>>>> :)
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com>
>> 于2020年3月10日周二
>> >>>> > > 下午5:49写道：
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
>> >>>> > released
>> >>>> > > > > > > > >>>>> Flink
>> >>>> > > > > > > > >>>>>>>>> 1.10.0. We
>> >>>> > > > > > > > >>>>>>>>>>>>>> already
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
>> >>>> improvements/bugs
>> >>>> > in
>> >>>> > > > > > > > >>>> the
>> >>>> > > > > > > > >>>>>>>>>>> release-1.10
>> >>>> > > > > > > > >>>>>>>>>>>>>>> branch,
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1
>> >>>> release
>> >>>> > > cycle.
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the
>> >>>> license and
>> >>>> > > > > > > > >>>> notice
>> >>>> > > > > > > > >>>>>>> file
>> >>>> > > > > > > > >>>>>>>> in
>> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource
>> leak
>> >>>> in
>> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should
>> be
>> >>>> known
>> >>>> > > in
>> >>>> > > > > > > > >>>>>>>>> HiveCatalog
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
>> >>>> ArrayIndexOufOBoundsException in
>> >>>> > > > > > > > >>>>>>>> TypeExtractor
>> >>>> > > > > > > > >>>>>>>>>>> when
>> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is
>> >>>> failed if
>> >>>> > > > > > > > >>>> running
>> >>>> > > > > > > > >>>>>> in
>> >>>> > > > > > > > >>>>>>>>> blink
>> >>>> > > > > > > > >>>>>>>>>>>>> planner
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints
>> >>>> are not
>> >>>> > > > > > > > >>>> reset
>> >>>> > > > > > > > >>>>>> on
>> >>>> > > > > > > > >>>>>>>> task
>> >>>> > > > > > > > >>>>>>>>>>>>> recovery
>> >>>> > > > > > > > >>>>>>>>>>>>>> in
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
>> function
>> >>>> using
>> >>>> > > sql
>> >>>> > > > > > > > >>>>>>> casuing
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation
>> >>>> failed
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following
>> >>>> issues
>> >>>> > > should
>> >>>> > > > > > > > >>>> be
>> >>>> > > > > > > > >>>>>>>> merged
>> >>>> > > > > > > > >>>>>>>>>>> before
>> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
>> >>>> issue):
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
>> >>>> Metaspace
>> >>>> > OOM
>> >>>> > > > > > > > >>>>> error
>> >>>> > > > > > > > >>>>>> on
>> >>>> > > > > > > > >>>>>>>>>>> repeated
>> >>>> > > > > > > > >>>>>>>>>>>>> job
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value
>> >>>> for JVM
>> >>>> > > > > > > > >>>>>> Metaspace
>> >>>> > > > > > > > >>>>>>> to
>> >>>> > > > > > > > >>>>>>>>>>>> minimise
>> >>>> > > > > > > > >>>>>>>>>>>>>> its
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces
>> >>>> wrong
>> >>>> > > > > > > > >>>>> aggregate
>> >>>> > > > > > > > >>>>>>>>> results
>> >>>> > > > > > > > >>>>>>>>>>> with
>> >>>> > > > > > > > >>>>>>>>>>>>>> state
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
>> >>>> extract
>> >>>> > > > > > > > >>>> correct
>> >>>> > > > > > > > >>>>>>>> unique
>> >>>> > > > > > > > >>>>>>>>> key
>> >>>> > > > > > > > >>>>>>>>>>>> for
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release
>> >>>> manager and
>> >>>> > > kick
>> >>>> > > > > > > > >>>>> off
>> >>>> > > > > > > > >>>>>>> the
>> >>>> > > > > > > > >>>>>>>>>>> release
>> >>>> > > > > > > > >>>>>>>>>>>>>>> process
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What
>> do
>> >>>> you
>> >>>> > > > > > > > >>>> think?
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
>> >>>> blocker
>> >>>> > > > > > > > >>>> issues
>> >>>> > > > > > > > >>>>>>> need
>> >>>> > > > > > > > >>>>>>>>> to be
>> >>>> > > > > > > > >>>>>>>>>>>>> fixed
>> >>>> > > > > > > > >>>>>>>>>>>>>> in
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>> --
>> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
>> >>>> > > > > > > > >>>>>>>>
>> >>>> > > > > > > > >>>>>>>
>> >>>> > > > > > > > >>>>>>
>> >>>> > > > > > > > >>>>>
>> >>>> > > > > > > > >>>>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > > >>
>> >>>> > > > > > > >
>> >>>> > > > > > > >
>> >>>> > > > > > >
>> >>>> > > > > >
>> >>>> > > > >
>> >>>> > > >
>> >>>> > >
>> >>>> >
>> >>>>
>> >>>
>>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by "DONG, Weike" <ky...@connect.hku.hk>.
Hi Yu,
Thanks for your updates. I am still working on the fix for FLINK-16626 and
it is expected to be completed by this Sunday after thorough testing.
Sincerely,
Weike
On Fri, Apr 3, 2020 at 8:43 PM Yu Li <ca...@gmail.com> wrote:
> Updates for 1.10.1 watched issues (we are in good progress and almost there
> to produce the first RC, thanks all for the efforts):
>
> * Blockers (3 left)
> - [Closed] FLINK-16018 Improve error reporting when submitting batch job
> (instead of AskTimeoutException)
> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> job submission
> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
> use flink-sql-connector-elasticsearch7
> - [Closed] FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> minimise its OutOfMemoryError
> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> - [PR reviewed] FLINK-16576 State inconsistency on restore with memory
> state backends
> - [Under Discussion] FLINK-16626 Exception encountered when cancelling a
> job in yarn per-job mode
> - [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before client
> can retrieve JobResult
> - [PR reviewed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
> throws UnsupportedOperationException
>
> * Critical (1 left)
> - [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> with state clean up
> - [Closed] FLINK-16070 Blink planner can not extract correct unique key
> for UpsertStreamTableSink
> - [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be
> handled as Fatal Error in TaskManager
> - [May Postpone] FLINK-16408 Bind user code class loader to lifetime of a
> slot
>
> Please let me know if you find any missing ones, thanks.
>
> Best Regards,
> Yu
>
>
> On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
>
> > Here comes the latest status of issues in 1.10.1 watch list:
> >
> > * Blockers (4 left)
> > - [Under Discussion] FLINK-16018 Improve error reporting when
> submitting
> > batch job (instead of AskTimeoutException)
> > - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> > repeated job submission
> > - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> when
> > use flink-sql-connector-elasticsearch7
> > - [PR Approved] FLINK-16262 Class loader problem with
> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > minimise its OutOfMemoryError
> > - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> > - [In Progress] FLINK-16576 State inconsistency on restore with memory
> > state backends
> > - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down MiniCluster
> > before client can retrieve JobResult
> >
> > * Critical (2 left)
> > - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> > with state clean up
> > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> > unique key for UpsertStreamTableSink
> > - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should be
> > handled as Fatal Error in TaskManager
> > - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
> >
> > There's a new blocker added. Please let me know if you find any other
> > issues should be put into the watch list. Thanks.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> >
> >> Hi All,
> >>
> >> Here is the status update of issues in 1.10.1 watch list:
> >>
> >> * Blockers (7)
> >>
> >> - [Under Discussion] FLINK-16018 Improve error reporting when
> >> submitting batch job (instead of AskTimeoutException)
> >>
> >> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
> error
> >> on repeated job submission
> >>
> >> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >> ClassNotFoundException when use flink-sql-connector-elasticsearch7
> >>
> >> - [PR Submitted] FLINK-16262 Class loader problem with
> >> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>
> >> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> >> minimise its OutOfMemoryError
> >>
> >> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> >>
> >> - [Open] FLINK-16576 State inconsistency on restore with memory state
> >> backends
> >>
> >>
> >> * Critical (4)
> >>
> >> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> >> with state clean up
> >>
> >> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> >> unique key for UpsertStreamTableSink
> >>
> >> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> >> handled as Fatal Error in TaskManager
> >>
> >> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
> >>
> >> Please let me know if you see any new blockers to add to the list.
> Thanks.
> >>
> >> Best Regards,
> >> Yu
> >>
> >>
> >> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> >>
> >>> Thanks for the updates Till!
> >>>
> >>> For FLINK-16018, maybe we could create two sub-tasks for easy and
> >>> complete fix separately, and only include the easy one in 1.10.1? Or
> please
> >>> just feel free to postpone the whole task to 1.10.2 if "all or nothing"
> >>> policy is preferred (smile). Thanks.
> >>>
> >>> Best Regards,
> >>> Yu
> >>>
> >>>
> >>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org>
> >>> wrote:
> >>>
> >>>> +1 for a soonish bug fix release. Thanks for volunteering as our
> release
> >>>> manager Yu.
> >>>>
> >>>> I think we can soon merge the increase of metaspace size and improving
> >>>> the
> >>>> error message. The assumption is that we currently don't have too many
> >>>> small Flink 1.10 deployments with a process size <= 1GB. Of course,
> the
> >>>> sooner we release the bug fix release, the fewer deployments will be
> >>>> affected by this change.
> >>>>
> >>>> For FLINK-16018, I think there would be an easy solution which we
> could
> >>>> include in the bug fix release. The proper fix will most likely take a
> >>>> bit
> >>>> longer, though.
> >>>>
> >>>> Cheers,
> >>>> Till
> >>>>
> >>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <azagrebin@apache.org
> >
> >>>> wrote:
> >>>>
> >>>> > > @Andrey and @Xintong - could we have a quick poll on the user
> >>>> mailing
> >>>> > list
> >>>> > > about increasing the metaspace size in Flink 1.10.1? Specifically
> >>>> asking
> >>>> > > for who has very small TM setups?
> >>>> >
> >>>> > There has been a survey about this topic since 10 days:
> >>>> >
> >>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
> >>>> > I can ask there about the small TM setups.
> >>>> >
> >>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
> >>>> >
> >>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on
> >>>> restore
> >>>> > with
> >>>> > > memory state backends
> >>>> > >
> >>>> > > Let me recompile the watching list with recent feedbacks. There're
> >>>> > totally
> >>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which
> 14
> >>>> > > already resolved and 31 left, and the below ones are watched
> >>>> (meaning
> >>>> > that
> >>>> > > once the below ones got fixed, we will kick of the releasing with
> >>>> left
> >>>> > ones
> >>>> > > postponed to 1.10.2 unless objections):
> >>>> > >
> >>>> > > * Blockers (7)
> >>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
> >>>> > submitting
> >>>> > > batch job (instead of AskTimeoutException)
> >>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace
> OOM
> >>>> error
> >>>> > > on repeated job submission
> >>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> >>>> > ClassNotFoundException
> >>>> > > when use flink-sql-connector-elasticsearch7
> >>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
> >>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> >>>> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace
> to
> >>>> > > minimise its OutOfMemoryError
> >>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
> >>>> > > - [Open] FLINK-16576 State inconsistency on restore with memory
> >>>> state
> >>>> > > backends
> >>>> > >
> >>>> > > * Critical (4)
> >>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate
> >>>> results
> >>>> > with
> >>>> > > state clean up
> >>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract
> correct
> >>>> > unique
> >>>> > > key for UpsertStreamTableSink
> >>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should
> be
> >>>> > > handled as Fatal Error in TaskManager
> >>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of
> a
> >>>> slot
> >>>> > >
> >>>> > > Please raise your hand if you find any other issues should be put
> >>>> into
> >>>> > this
> >>>> > > list, thanks.
> >>>> > >
> >>>> > > Please also note that 1.10.2 version is already created (thanks
> for
> >>>> the
> >>>> > > help @jincheng), and feel free to adjust/assign fix version to it
> >>>> when
> >>>> > > necessary.
> >>>> > >
> >>>> > > Best Regards,
> >>>> > > Yu
> >>>> > >
> >>>> > >
> >>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org>
> >>>> wrote:
> >>>> > >
> >>>> > > > @Andrey and @Xintong - could we have a quick poll on the user
> >>>> mailing
> >>>> > > list
> >>>> > > > about increasing the metaspace size in Flink 1.10.1?
> Specifically
> >>>> > asking
> >>>> > > > for who has very small TM setups?
> >>>> > > >
> >>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
> >>>> > > >
> >>>> > > > > Thanks for the reminder Stephan and the inputs/discussion
> >>>> Andrey and
> >>>> > > > > Xintong. I totally agree that we should try best not to
> >>>> introduce new
> >>>> > > > > issues when resolving the existing one. Will watch closely
> >>>> about the
> >>>> > > > > discussion and conclusion around metaspace configuration.
> >>>> > > > >
> >>>> > > > > From the point as a developer instead of RM, I'm optimistic
> that
> >>>> > > > reserving
> >>>> > > > > 160MB more memory to JVM metaspace by default won't cause a
> big
> >>>> > > problem,
> >>>> > > > > based on the assumption that nowadays assigning 2GB/4GB memory
> >>>> per TM
> >>>> > > is
> >>>> > > > > the common case.
> >>>> > > > >
> >>>> > > > > Best Regards,
> >>>> > > > > Yu
> >>>> > > > >
> >>>> > > > >
> >>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
> >>>> tonysong820@gmail.com>
> >>>> > > > wrote:
> >>>> > > > >
> >>>> > > > > > I think in FLINK-16406 we have not only increased the
> default
> >>>> value
> >>>> > > for
> >>>> > > > > > metaspace, but also increased the 'process.size' in default
> >>>> > > > > > 'flink-conf.yaml'.
> >>>> > > > > >
> >>>> > > > > > That means:
> >>>> > > > > >
> >>>> > > > > > - For people using the default 'flink-conf.yaml', it's
> not
> >>>> > really
> >>>> > > > > taking
> >>>> > > > > > memory from heap / managed for metaspace.
> >>>> > > > > > - For people using custom configuration files:
> >>>> > > > > > - If they are specifying 'flink.size', then metaspace
> >>>> does
> >>>> > not
> >>>> > > > > really
> >>>> > > > > > affect the flink memory internal components (heap,
> >>>> managed,
> >>>> > > > etc.).
> >>>> > > > > > - If they are specifying 'process.size' (most likely
> >>>> active
> >>>> > > > > setups),
> >>>> > > > > > then they will have less heap / managed memory, or
> >>>> worse,
> >>>> > their
> >>>> > > > > > configuration may not work without adjustment.
> >>>> > > > > > - For a TM with large process memory (e.g., 4 GB or
> >>>> more),
> >>>> > > it
> >>>> > > > is
> >>>> > > > > > probably ok that the metaspace takes 160 MB more
> >>>> memory.
> >>>> > > > > > - For a TM with smaller total memory (1 or 2 GB)
> the
> >>>> > > metaspace
> >>>> > > > > > increase is more likely to cause problem.
> >>>> > > > > >
> >>>> > > > > > So basically only people have small 'process.size' in custom
> >>>> config
> >>>> > > > file
> >>>> > > > > > are really affected. I'm not sure what is the proportion of
> >>>> such
> >>>> > use
> >>>> > > > > cases
> >>>> > > > > > though. (From questions asked on the user ML, probably not
> >>>> much).
> >>>> > > > > >
> >>>> > > > > > Thank you~
> >>>> > > > > >
> >>>> > > > > > Xintong Song
> >>>> > > > > >
> >>>> > > > > >
> >>>> > > > > >
> >>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
> >>>> sewen@apache.org>
> >>>> > > > wrote:
> >>>> > > > > >
> >>>> > > > > > > No need to revert it now - I am not saying it should not
> go
> >>>> into
> >>>> > > > > 1.10.1,
> >>>> > > > > > I
> >>>> > > > > > > am just saying this is not clear to me yet.
> >>>> > > > > > >
> >>>> > > > > > > We have to trade off the fact that we may break some
> >>>> deployments
> >>>> > > with
> >>>> > > > > the
> >>>> > > > > > > fact that we will "safe" a lot of new deployments.
> >>>> > > > > > > I simply lack the data points / insight at the moment to
> >>>> > understand
> >>>> > > > how
> >>>> > > > > > > significant both cases are, meaning how many users would
> be
> >>>> > > affected
> >>>> > > > > and
> >>>> > > > > > > how badly.
> >>>> > > > > > >
> >>>> > > > > > > Independent of that, improving the error message is always
> >>>> > helpful.
> >>>> > > > > > >
> >>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> >>>> > > > > > > azagrebin.apache@gmail.com>
> >>>> > > > > > > wrote:
> >>>> > > > > > >
> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
> >>>> expect
> >>>> > > to
> >>>> > > > > > > upgrade
> >>>> > > > > > > > > that without config adjustments. That might not be
> >>>> possible
> >>>> > > with
> >>>> > > > > that
> >>>> > > > > > > > > change.
> >>>> > > > > > > >
> >>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try
> to
> >>>> > > improve
> >>>> > > > > > error
> >>>> > > > > > > > message and docs.
> >>>> > > > > > > >
> >>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
> >>>> sewen@apache.org>
> >>>> > > wrote:
> >>>> > > > > > > > >
> >>>> > > > > > > > > @Andrey about the increase in metaspace size
> >>>> > > > > > > > > - I have no concerns for 1.11.0.
> >>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
> >>>> expect
> >>>> > > to
> >>>> > > > > > > upgrade
> >>>> > > > > > > > > that without config adjustments. That might not be
> >>>> possible
> >>>> > > with
> >>>> > > > > that
> >>>> > > > > > > > > change.
> >>>> > > > > > > > >
> >>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> >>>> > > > > > > > azagrebin.apache@gmail.com>
> >>>> > > > > > > > > wrote:
> >>>> > > > > > > > >
> >>>> > > > > > > > >>
> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
> >>>> error
> >>>> > on
> >>>> > > > > > repeated
> >>>> > > > > > > > >> job”
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> My understanding that the issue is basically covered
> >>>> by:
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
> >>>> handled as
> >>>> > > > Fatal
> >>>> > > > > > > Error
> >>>> > > > > > > > >> in TaskManager
> >>>> > > > > > > > >> no full consensus there but improving error message
> >>>> for
> >>>> > > > existing
> >>>> > > > > > > task
> >>>> > > > > > > > >> thread fatal handling could be done at least
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
> >>>> Metaspace to
> >>>> > > > > minimise
> >>>> > > > > > > its
> >>>> > > > > > > > >> OutOfMemoryError
> >>>> > > > > > > > >> see further
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport"
> from
> >>>> > > > dynamically
> >>>> > > > > > > > loaded
> >>>> > > > > > > > >> AWS connectors
> >>>> > > > > > > > >> not sure whether this is a blocker but looks close
> to
> >>>> be
> >>>> > > > resolved
> >>>> > > > > > > > >>
> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> >>>> > Metaspace"
> >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
> >>>> release?
> >>>> > > It
> >>>> > > > > > > changes
> >>>> > > > > > > > >>> setups, takes away memory from heap / managed memory
> >>>> on
> >>>> > > > existing
> >>>> > > > > > > setups
> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> My understanding was that increasing to 256m resolved
> >>>> the
> >>>> > > > reported
> >>>> > > > > > > > problems
> >>>> > > > > > > > >> and we decided to make the change so I have merged it
> >>>> today
> >>>> > as
> >>>> > > > > there
> >>>> > > > > > > > were
> >>>> > > > > > > > >> no more concerns.
> >>>> > > > > > > > >> If there are concerns I can revert it.
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> On the other hand, I think improving the message
> error
> >>>> with
> >>>> > > > > > reference
> >>>> > > > > > > to
> >>>> > > > > > > > >> the metaspace option should help the most
> >>>> > > > > > > > >> because user would not have to read all docs to fix
> it
> >>>> > > > > > > > >> then maybe this change is not even needed.
> >>>> > > > > > > > >>
> >>>> > > > > > > > >> Best,
> >>>> > > > > > > > >> Andrey
> >>>> > > > > > > > >>
> >>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
> >>>> sewen@apache.org>
> >>>> > > > wrote:
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
> >>>> error
> >>>> > on
> >>>> > > > > > repeated
> >>>> > > > > > > > >> job"
> >>>> > > > > > > > >>> - I don't think we have consensus on the exact
> >>>> solution,
> >>>> > yet,
> >>>> > > > and
> >>>> > > > > > > some
> >>>> > > > > > > > >> of
> >>>> > > > > > > > >>> the changes might also have side effects that are
> >>>> hard to
> >>>> > > > > predict,
> >>>> > > > > > > so I
> >>>> > > > > > > > >> am
> >>>> > > > > > > > >>> not sure we should rush this in.
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> >>>> > Metaspace"
> >>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
> >>>> release?
> >>>> > > It
> >>>> > > > > > > changes
> >>>> > > > > > > > >>> setups, takes away memory from heap / managed memory
> >>>> on
> >>>> > > > existing
> >>>> > > > > > > setups
> >>>> > > > > > > > >>> that keep their flink-conf.yaml.
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>> We may need to unblock the release form these two
> >>>> issues
> >>>> > and
> >>>> > > > > think
> >>>> > > > > > > > about
> >>>> > > > > > > > >>> having 1.10.2 in the near future.
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
> >>>> carp84@gmail.com>
> >>>> > > > wrote:
> >>>> > > > > > > > >>>
> >>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on
> >>>> these
> >>>> > two.
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>>> Best Regards,
> >>>> > > > > > > > >>>> Yu
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
> >>>> imjark@gmail.com>
> >>>> > > > wrote:
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
> >>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>> From the Table SQL module, I think we should also
> >>>> try to
> >>>> > > get
> >>>> > > > in
> >>>> > > > > > the
> >>>> > > > > > > > >>>>> following issues:
> >>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
> >>>> > > parameters
> >>>> > > > > from
> >>>> > > > > > > SQL
> >>>> > > > > > > > >> CLI
> >>>> > > > > > > > >>>>> environment
> >>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
> >>>> watermark
> >>>> > > > interval,
> >>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> >>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any
> uncatched
> >>>> > > > exception
> >>>> > > > > is
> >>>> > > > > > > > >> thrown
> >>>> > > > > > > > >>>>> this will improve much experience when using SQL
> CLI
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>> Best,
> >>>> > > > > > > > >>>>> Jark
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
> >>>> carp84@gmail.com>
> >>>> > > > wrote:
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added
> >>>> 1.10.1 into
> >>>> > > > > > > FLINK-16225
> >>>> > > > > > > > >>>> fix
> >>>> > > > > > > > >>>>>> versions and promoted its priority to Critical.
> >>>> Will
> >>>> > also
> >>>> > > > > watch
> >>>> > > > > > > the
> >>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>> Best Regards,
> >>>> > > > > > > > >>>>>> Yu
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> >>>> > > > > > > azagrebin@apache.org
> >>>> > > > > > > > >
> >>>> > > > > > > > >>>>>> wrote:
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>>> Hi Yu,
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
> >>>> discussion!
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> Apart from
> >>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
> >>>> Metaspace
> >>>> > to
> >>>> > > > > > > minimise
> >>>> > > > > > > > >>>> its
> >>>> > > > > > > > >>>>>>> OutOfMemoryError
> >>>> > > > > > > > >>>>>>> which should be merged soon
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> I think we should also try to get in the
> following
> >>>> > > issues:
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should
> be
> >>>> > handled
> >>>> > > > as
> >>>> > > > > > > Fatal
> >>>> > > > > > > > >>>>> Error
> >>>> > > > > > > > >>>>>> in
> >>>> > > > > > > > >>>>>>> TaskManager
> >>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even in
> a
> >>>> > better
> >>>> > > > way
> >>>> > > > > > > > because
> >>>> > > > > > > > >>>>> OOM
> >>>> > > > > > > > >>>>>>> failure should point users to the docs
> immediately
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
> >>>> lifetime
> >>>> > > of
> >>>> > > > a
> >>>> > > > > > slot
> >>>> > > > > > > > >>>>>>> This should give a better protection against
> class
> >>>> > > loading
> >>>> > > > > > leaks
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
> >>>> submitting
> >>>> > > > batch
> >>>> > > > > > job
> >>>> > > > > > > > >>>>>> (instead
> >>>> > > > > > > > >>>>>>> of AskTimeoutException)
> >>>> > > > > > > > >>>>>>> This problem has recently happened for multiple
> >>>> users
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> Best,
> >>>> > > > > > > > >>>>>>> Andrey
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> >>>> > > > > > > > jingsonglee0@gmail.com>
> >>>> > > > > > > > >>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the
> >>>> 1.10.1
> >>>> > > > release.
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>> Some issues are very important, Users are
> looking
> >>>> > > forward
> >>>> > > > to
> >>>> > > > > > > them.
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>> Best,
> >>>> > > > > > > > >>>>>>>> Jingsong Lee
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> >>>> > > > > > karmagyz@gmail.com>
> >>>> > > > > > > > >>>>> wrote:
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>> Best,
> >>>> > > > > > > > >>>>>>>>> Yangze Guo
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> >>>> > > > > > > > >>>>> tonysong820@gmail.com>
> >>>> > > > > > > > >>>>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>> Yu,
> >>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
> >>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get
> some
> >>>> > inputs
> >>>> > > > for
> >>>> > > > > > > > >>>>>>> prioritizing
> >>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to
> >>>> me.
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>> Thank you~
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>> Xintong Song
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> >>>> > > > carp84@gmail.com>
> >>>> > > > > > > > >>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
> >>>> release
> >>>> > of
> >>>> > > a
> >>>> > > > > > major
> >>>> > > > > > > > >>>>>>> version
> >>>> > > > > > > > >>>>>>>>> will
> >>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
> >>>> depending on
> >>>> > > the
> >>>> > > > > > speed
> >>>> > > > > > > > >>>> of
> >>>> > > > > > > > >>>>>>>> blocker
> >>>> > > > > > > > >>>>>>>>>>> issue resolving:
> >>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
> >>>> > released
> >>>> > > > on
> >>>> > > > > > Jul
> >>>> > > > > > > > >>>>> 3rd
> >>>> > > > > > > > >>>>>>> [2]
> >>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
> >>>> > released
> >>>> > > > on
> >>>> > > > > > Oct
> >>>> > > > > > > > >>>>> 19th
> >>>> > > > > > > > >>>>>>> [4]
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of
> course,
> >>>> but
> >>>> > > could
> >>>> > > > > use
> >>>> > > > > > > > >>>> it
> >>>> > > > > > > > >>>>>> as a
> >>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me
> >>>> know if
> >>>> > any
> >>>> > > > > > > > >>>> concerns
> >>>> > > > > > > > >>>>>>>> Xintong.
> >>>> > > > > > > > >>>>>>>>>>> Thanks.
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> Best Regards,
> >>>> > > > > > > > >>>>>>>>>>> Yu
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> [1]
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>
> >>>> > > > > > > >
> >>>> > > > > > >
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >>>> > > > > > > > >>>>>>>>>>> [2]
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>
> >>>> > > > > > > >
> >>>> > > > > > >
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >>>> > > > > > > > >>>>>>>>>>> [3]
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>
> >>>> > > > > > > >
> >>>> > > > > > >
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >>>> > > > > > > > >>>>>>>>>>> [4]
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>
> >>>> > > > > > > >
> >>>> > > > > > >
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> >>>> > > > > > > > >>>>>> tonysong820@gmail.com>
> >>>> > > > > > > > >>>>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and
> volunteering
> >>>> to be
> >>>> > > the
> >>>> > > > > > > > >>>>> release
> >>>> > > > > > > > >>>>>>>>> manager.
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan
> for
> >>>> > > 1.10.1?
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> Thank you~
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> Xintong Song
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> >>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid>
> wrote:
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>> Best,
> >>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > >
> >>>> > ------------------------------------------------------------------
> >>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> >>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink
> 1.10.1
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will
> >>>> also
> >>>> > keep
> >>>> > > > an
> >>>> > > > > > > > >>>> eye
> >>>> > > > > > > > >>>>> on
> >>>> > > > > > > > >>>>>>>> these
> >>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
> >>>> > > > > > > > >>>>>>>>>>>>> Yu
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng
> <
> >>>> > > > > > > > >>>>> hequn@apache.org>
> >>>> > > > > > > > >>>>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion
> and
> >>>> > > > volunteer
> >>>> > > > > > > > >>>> as
> >>>> > > > > > > > >>>>>> the
> >>>> > > > > > > > >>>>>>>>> release
> >>>> > > > > > > > >>>>>>>>>>>>>> manager!
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1]
> >>>> which are
> >>>> > > > > marked
> >>>> > > > > > > > >>>>> as
> >>>> > > > > > > > >>>>>> a
> >>>> > > > > > > > >>>>>>>>> blocker:
> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year
> in
> >>>> > NOTICE
> >>>> > > > > files
> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> >>>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
> >>>> and
> >>>> > > usrlib
> >>>> > > > > > > > >>>>>> directory
> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> >>>> > > > > > > > >>>>> ClassNotFoundException
> >>>> > > > > > > > >>>>>>> when
> >>>> > > > > > > > >>>>>>>>> use
> >>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting
> when
> >>>> > > > submitting
> >>>> > > > > > > > >>>>> batch
> >>>> > > > > > > > >>>>>>> job
> >>>> > > > > > > > >>>>>>>>>>>> (instead
> >>>> > > > > > > > >>>>>>>>>>>>> of
> >>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in
> >>>> 1.10.1.
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> Best,
> >>>> > > > > > > > >>>>>>>>>>>>>> Hequn
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> [1]
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > >
> >>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> >>>> > > > > > > > >>>> carp84@gmail.com>
> >>>> > > > > > > > >>>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
> >>>> Thanks a
> >>>> > > lot!
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
> >>>> > > > > > > > >>>>>>>>>>>>>>> Yu
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng
> >>>> sun <
> >>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu.
> I
> >>>> would
> >>>> > > > like
> >>>> > > > > > > > >>>>> to
> >>>> > > > > > > > >>>>>>> give
> >>>> > > > > > > > >>>>>>>>> you a
> >>>> > > > > > > > >>>>>>>>>>>>> hand
> >>>> > > > > > > > >>>>>>>>>>>>>> at
> >>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
> >>>> finished.(If you
> >>>> > > > > > > > >>>> need)
> >>>> > > > > > > > >>>>>> :)
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
> >>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
> >>>> > > 下午5:49写道：
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
> >>>> > released
> >>>> > > > > > > > >>>>> Flink
> >>>> > > > > > > > >>>>>>>>> 1.10.0. We
> >>>> > > > > > > > >>>>>>>>>>>>>> already
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
> >>>> improvements/bugs
> >>>> > in
> >>>> > > > > > > > >>>> the
> >>>> > > > > > > > >>>>>>>>>>> release-1.10
> >>>> > > > > > > > >>>>>>>>>>>>>>> branch,
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1
> >>>> release
> >>>> > > cycle.
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the
> >>>> license and
> >>>> > > > > > > > >>>> notice
> >>>> > > > > > > > >>>>>>> file
> >>>> > > > > > > > >>>>>>>> in
> >>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> module
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource
> leak
> >>>> in
> >>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should
> be
> >>>> known
> >>>> > > in
> >>>> > > > > > > > >>>>>>>>> HiveCatalog
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
> >>>> ArrayIndexOufOBoundsException in
> >>>> > > > > > > > >>>>>>>> TypeExtractor
> >>>> > > > > > > > >>>>>>>>>>> when
> >>>> > > > > > > > >>>>>>>>>>>>>>> mapping
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is
> >>>> failed if
> >>>> > > > > > > > >>>> running
> >>>> > > > > > > > >>>>>> in
> >>>> > > > > > > > >>>>>>>>> blink
> >>>> > > > > > > > >>>>>>>>>>>>> planner
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints
> >>>> are not
> >>>> > > > > > > > >>>> reset
> >>>> > > > > > > > >>>>>> on
> >>>> > > > > > > > >>>>>>>> task
> >>>> > > > > > > > >>>>>>>>>>>>> recovery
> >>>> > > > > > > > >>>>>>>>>>>>>> in
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf
> function
> >>>> using
> >>>> > > sql
> >>>> > > > > > > > >>>>>>> casuing
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation
> >>>> failed
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following
> >>>> issues
> >>>> > > should
> >>>> > > > > > > > >>>> be
> >>>> > > > > > > > >>>>>>>> merged
> >>>> > > > > > > > >>>>>>>>>>> before
> >>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
> >>>> issue):
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
> >>>> Metaspace
> >>>> > OOM
> >>>> > > > > > > > >>>>> error
> >>>> > > > > > > > >>>>>> on
> >>>> > > > > > > > >>>>>>>>>>> repeated
> >>>> > > > > > > > >>>>>>>>>>>>> job
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value
> >>>> for JVM
> >>>> > > > > > > > >>>>>> Metaspace
> >>>> > > > > > > > >>>>>>> to
> >>>> > > > > > > > >>>>>>>>>>>> minimise
> >>>> > > > > > > > >>>>>>>>>>>>>> its
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces
> >>>> wrong
> >>>> > > > > > > > >>>>> aggregate
> >>>> > > > > > > > >>>>>>>>> results
> >>>> > > > > > > > >>>>>>>>>>> with
> >>>> > > > > > > > >>>>>>>>>>>>>> state
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
> >>>> extract
> >>>> > > > > > > > >>>> correct
> >>>> > > > > > > > >>>>>>>> unique
> >>>> > > > > > > > >>>>>>>>> key
> >>>> > > > > > > > >>>>>>>>>>>> for
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release
> >>>> manager and
> >>>> > > kick
> >>>> > > > > > > > >>>>> off
> >>>> > > > > > > > >>>>>>> the
> >>>> > > > > > > > >>>>>>>>>>> release
> >>>> > > > > > > > >>>>>>>>>>>>>>> process
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What
> do
> >>>> you
> >>>> > > > > > > > >>>> think?
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
> >>>> blocker
> >>>> > > > > > > > >>>> issues
> >>>> > > > > > > > >>>>>>> need
> >>>> > > > > > > > >>>>>>>>> to be
> >>>> > > > > > > > >>>>>>>>>>>>> fixed
> >>>> > > > > > > > >>>>>>>>>>>>>> in
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> >>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
> >>>> > > > > > > > >>>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>>>
> >>>> > > > > > > > >>>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>> --
> >>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
> >>>> > > > > > > > >>>>>>>>
> >>>> > > > > > > > >>>>>>>
> >>>> > > > > > > > >>>>>>
> >>>> > > > > > > > >>>>>
> >>>> > > > > > > > >>>>
> >>>> > > > > > > > >>
> >>>> > > > > > > > >>
> >>>> > > > > > > >
> >>>> > > > > > > >
> >>>> > > > > > >
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Updates for 1.10.1 watched issues (we are in good progress and almost there
to produce the first RC, thanks all for the efforts):
* Blockers (3 left)
- [Closed] FLINK-16018 Improve error reporting when submitting batch job
(instead of AskTimeoutException)
- [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
job submission
- [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
use flink-sql-connector-elasticsearch7
- [Closed] FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- [Closed] FLINK-16406 Increase default value for JVM Metaspace to
minimise its OutOfMemoryError
- [Closed] FLINK-16454 Update the copyright year in NOTICE files
- [PR reviewed] FLINK-16576 State inconsistency on restore with memory
state backends
- [Under Discussion] FLINK-16626 Exception encountered when cancelling a
job in yarn per-job mode
- [Closed] FLINK-16705 LocalExecutor tears down MiniCluster before client
can retrieve JobResult
- [PR reviewed] FLINK-16913 ReadableConfigToConfigurationAdapter#getEnum
throws UnsupportedOperationException
* Critical (1 left)
- [Closed] FLINK-14316 stuck in "Job leader ... lost leadership" error
- [Closed] FLINK-16047 Blink planner produces wrong aggregate results
with state clean up
- [Closed] FLINK-16070 Blink planner can not extract correct unique key
for UpsertStreamTableSink
- [Fix for 1.10.1 is Done] FLINK-16225 Metaspace Out Of Memory should be
handled as Fatal Error in TaskManager
- [May Postpone] FLINK-16408 Bind user code class loader to lifetime of a
slot
Please let me know if you find any missing ones, thanks.
Best Regards,
Yu
On Fri, 27 Mar 2020 at 21:06, Yu Li <ca...@gmail.com> wrote:
> Here comes the latest status of issues in 1.10.1 watch list:
>
> * Blockers (4 left)
> - [Under Discussion] FLINK-16018 Improve error reporting when submitting
> batch job (instead of AskTimeoutException)
> - [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on
> repeated job submission
> - [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
> use flink-sql-connector-elasticsearch7
> - [PR Approved] FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> minimise its OutOfMemoryError
> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
> - [In Progress] FLINK-16576 State inconsistency on restore with memory
> state backends
> - [New] [PR Submitted] FLINK-16705 LocalExecutor tears down MiniCluster
> before client can retrieve JobResult
>
> * Critical (2 left)
> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> with state clean up
> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> unique key for UpsertStreamTableSink
> - [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should be
> handled as Fatal Error in TaskManager
> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
>
> There's a new blocker added. Please let me know if you find any other
> issues should be put into the watch list. Thanks.
>
> Best Regards,
> Yu
>
>
> On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
>
>> Hi All,
>>
>> Here is the status update of issues in 1.10.1 watch list:
>>
>> * Blockers (7)
>>
>> - [Under Discussion] FLINK-16018 Improve error reporting when
>> submitting batch job (instead of AskTimeoutException)
>>
>> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
>> on repeated job submission
>>
>> - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> ClassNotFoundException when use flink-sql-connector-elasticsearch7
>>
>> - [PR Submitted] FLINK-16262 Class loader problem with
>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>
>> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> minimise its OutOfMemoryError
>>
>> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>>
>> - [Open] FLINK-16576 State inconsistency on restore with memory state
>> backends
>>
>>
>> * Critical (4)
>>
>> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
>> with state clean up
>>
>> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>> unique key for UpsertStreamTableSink
>>
>> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
>> handled as Fatal Error in TaskManager
>>
>> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
>>
>> Please let me know if you see any new blockers to add to the list. Thanks.
>>
>> Best Regards,
>> Yu
>>
>>
>> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
>>
>>> Thanks for the updates Till!
>>>
>>> For FLINK-16018, maybe we could create two sub-tasks for easy and
>>> complete fix separately, and only include the easy one in 1.10.1? Or please
>>> just feel free to postpone the whole task to 1.10.2 if "all or nothing"
>>> policy is preferred (smile). Thanks.
>>>
>>> Best Regards,
>>> Yu
>>>
>>>
>>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org>
>>> wrote:
>>>
>>>> +1 for a soonish bug fix release. Thanks for volunteering as our release
>>>> manager Yu.
>>>>
>>>> I think we can soon merge the increase of metaspace size and improving
>>>> the
>>>> error message. The assumption is that we currently don't have too many
>>>> small Flink 1.10 deployments with a process size <= 1GB. Of course, the
>>>> sooner we release the bug fix release, the fewer deployments will be
>>>> affected by this change.
>>>>
>>>> For FLINK-16018, I think there would be an easy solution which we could
>>>> include in the bug fix release. The proper fix will most likely take a
>>>> bit
>>>> longer, though.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <az...@apache.org>
>>>> wrote:
>>>>
>>>> > > @Andrey and @Xintong - could we have a quick poll on the user
>>>> mailing
>>>> > list
>>>> > > about increasing the metaspace size in Flink 1.10.1? Specifically
>>>> asking
>>>> > > for who has very small TM setups?
>>>> >
>>>> > There has been a survey about this topic since 10 days:
>>>> >
>>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
>>>> > I can ask there about the small TM setups.
>>>> >
>>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
>>>> >
>>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on
>>>> restore
>>>> > with
>>>> > > memory state backends
>>>> > >
>>>> > > Let me recompile the watching list with recent feedbacks. There're
>>>> > totally
>>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
>>>> > > already resolved and 31 left, and the below ones are watched
>>>> (meaning
>>>> > that
>>>> > > once the below ones got fixed, we will kick of the releasing with
>>>> left
>>>> > ones
>>>> > > postponed to 1.10.2 unless objections):
>>>> > >
>>>> > > * Blockers (7)
>>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
>>>> > submitting
>>>> > > batch job (instead of AskTimeoutException)
>>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
>>>> error
>>>> > > on repeated job submission
>>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
>>>> > ClassNotFoundException
>>>> > > when use flink-sql-connector-elasticsearch7
>>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
>>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>>> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>>>> > > minimise its OutOfMemoryError
>>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
>>>> > > - [Open] FLINK-16576 State inconsistency on restore with memory
>>>> state
>>>> > > backends
>>>> > >
>>>> > > * Critical (4)
>>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate
>>>> results
>>>> > with
>>>> > > state clean up
>>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>>>> > unique
>>>> > > key for UpsertStreamTableSink
>>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
>>>> > > handled as Fatal Error in TaskManager
>>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
>>>> slot
>>>> > >
>>>> > > Please raise your hand if you find any other issues should be put
>>>> into
>>>> > this
>>>> > > list, thanks.
>>>> > >
>>>> > > Please also note that 1.10.2 version is already created (thanks for
>>>> the
>>>> > > help @jincheng), and feel free to adjust/assign fix version to it
>>>> when
>>>> > > necessary.
>>>> > >
>>>> > > Best Regards,
>>>> > > Yu
>>>> > >
>>>> > >
>>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org>
>>>> wrote:
>>>> > >
>>>> > > > @Andrey and @Xintong - could we have a quick poll on the user
>>>> mailing
>>>> > > list
>>>> > > > about increasing the metaspace size in Flink 1.10.1? Specifically
>>>> > asking
>>>> > > > for who has very small TM setups?
>>>> > > >
>>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
>>>> > > >
>>>> > > > > Thanks for the reminder Stephan and the inputs/discussion
>>>> Andrey and
>>>> > > > > Xintong. I totally agree that we should try best not to
>>>> introduce new
>>>> > > > > issues when resolving the existing one. Will watch closely
>>>> about the
>>>> > > > > discussion and conclusion around metaspace configuration.
>>>> > > > >
>>>> > > > > From the point as a developer instead of RM, I'm optimistic that
>>>> > > > reserving
>>>> > > > > 160MB more memory to JVM metaspace by default won't cause a big
>>>> > > problem,
>>>> > > > > based on the assumption that nowadays assigning 2GB/4GB memory
>>>> per TM
>>>> > > is
>>>> > > > > the common case.
>>>> > > > >
>>>> > > > > Best Regards,
>>>> > > > > Yu
>>>> > > > >
>>>> > > > >
>>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
>>>> tonysong820@gmail.com>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > > > I think in FLINK-16406 we have not only increased the default
>>>> value
>>>> > > for
>>>> > > > > > metaspace, but also increased the 'process.size' in default
>>>> > > > > > 'flink-conf.yaml'.
>>>> > > > > >
>>>> > > > > > That means:
>>>> > > > > >
>>>> > > > > > - For people using the default 'flink-conf.yaml', it's not
>>>> > really
>>>> > > > > taking
>>>> > > > > > memory from heap / managed for metaspace.
>>>> > > > > > - For people using custom configuration files:
>>>> > > > > > - If they are specifying 'flink.size', then metaspace
>>>> does
>>>> > not
>>>> > > > > really
>>>> > > > > > affect the flink memory internal components (heap,
>>>> managed,
>>>> > > > etc.).
>>>> > > > > > - If they are specifying 'process.size' (most likely
>>>> active
>>>> > > > > setups),
>>>> > > > > > then they will have less heap / managed memory, or
>>>> worse,
>>>> > their
>>>> > > > > > configuration may not work without adjustment.
>>>> > > > > > - For a TM with large process memory (e.g., 4 GB or
>>>> more),
>>>> > > it
>>>> > > > is
>>>> > > > > > probably ok that the metaspace takes 160 MB more
>>>> memory.
>>>> > > > > > - For a TM with smaller total memory (1 or 2 GB) the
>>>> > > metaspace
>>>> > > > > > increase is more likely to cause problem.
>>>> > > > > >
>>>> > > > > > So basically only people have small 'process.size' in custom
>>>> config
>>>> > > > file
>>>> > > > > > are really affected. I'm not sure what is the proportion of
>>>> such
>>>> > use
>>>> > > > > cases
>>>> > > > > > though. (From questions asked on the user ML, probably not
>>>> much).
>>>> > > > > >
>>>> > > > > > Thank you~
>>>> > > > > >
>>>> > > > > > Xintong Song
>>>> > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
>>>> sewen@apache.org>
>>>> > > > wrote:
>>>> > > > > >
>>>> > > > > > > No need to revert it now - I am not saying it should not go
>>>> into
>>>> > > > > 1.10.1,
>>>> > > > > > I
>>>> > > > > > > am just saying this is not clear to me yet.
>>>> > > > > > >
>>>> > > > > > > We have to trade off the fact that we may break some
>>>> deployments
>>>> > > with
>>>> > > > > the
>>>> > > > > > > fact that we will "safe" a lot of new deployments.
>>>> > > > > > > I simply lack the data points / insight at the moment to
>>>> > understand
>>>> > > > how
>>>> > > > > > > significant both cases are, meaning how many users would be
>>>> > > affected
>>>> > > > > and
>>>> > > > > > > how badly.
>>>> > > > > > >
>>>> > > > > > > Independent of that, improving the error message is always
>>>> > helpful.
>>>> > > > > > >
>>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>>>> > > > > > > azagrebin.apache@gmail.com>
>>>> > > > > > > wrote:
>>>> > > > > > >
>>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>>>> expect
>>>> > > to
>>>> > > > > > > upgrade
>>>> > > > > > > > > that without config adjustments. That might not be
>>>> possible
>>>> > > with
>>>> > > > > that
>>>> > > > > > > > > change.
>>>> > > > > > > >
>>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
>>>> > > improve
>>>> > > > > > error
>>>> > > > > > > > message and docs.
>>>> > > > > > > >
>>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <
>>>> sewen@apache.org>
>>>> > > wrote:
>>>> > > > > > > > >
>>>> > > > > > > > > @Andrey about the increase in metaspace size
>>>> > > > > > > > > - I have no concerns for 1.11.0.
>>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>>>> expect
>>>> > > to
>>>> > > > > > > upgrade
>>>> > > > > > > > > that without config adjustments. That might not be
>>>> possible
>>>> > > with
>>>> > > > > that
>>>> > > > > > > > > change.
>>>> > > > > > > > >
>>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
>>>> > > > > > > > azagrebin.apache@gmail.com>
>>>> > > > > > > > > wrote:
>>>> > > > > > > > >
>>>> > > > > > > > >>
>>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>>>> error
>>>> > on
>>>> > > > > > repeated
>>>> > > > > > > > >> job”
>>>> > > > > > > > >>
>>>> > > > > > > > >> My understanding that the issue is basically covered
>>>> by:
>>>> > > > > > > > >>
>>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
>>>> handled as
>>>> > > > Fatal
>>>> > > > > > > Error
>>>> > > > > > > > >> in TaskManager
>>>> > > > > > > > >> no full consensus there but improving error message
>>>> for
>>>> > > > existing
>>>> > > > > > > task
>>>> > > > > > > > >> thread fatal handling could be done at least
>>>> > > > > > > > >>
>>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
>>>> Metaspace to
>>>> > > > > minimise
>>>> > > > > > > its
>>>> > > > > > > > >> OutOfMemoryError
>>>> > > > > > > > >> see further
>>>> > > > > > > > >>
>>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
>>>> > > > dynamically
>>>> > > > > > > > loaded
>>>> > > > > > > > >> AWS connectors
>>>> > > > > > > > >> not sure whether this is a blocker but looks close to
>>>> be
>>>> > > > resolved
>>>> > > > > > > > >>
>>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>>>> > Metaspace"
>>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>>>> release?
>>>> > > It
>>>> > > > > > > changes
>>>> > > > > > > > >>> setups, takes away memory from heap / managed memory
>>>> on
>>>> > > > existing
>>>> > > > > > > setups
>>>> > > > > > > > >>> that keep their flink-conf.yaml.
>>>> > > > > > > > >>
>>>> > > > > > > > >> My understanding was that increasing to 256m resolved
>>>> the
>>>> > > > reported
>>>> > > > > > > > problems
>>>> > > > > > > > >> and we decided to make the change so I have merged it
>>>> today
>>>> > as
>>>> > > > > there
>>>> > > > > > > > were
>>>> > > > > > > > >> no more concerns.
>>>> > > > > > > > >> If there are concerns I can revert it.
>>>> > > > > > > > >>
>>>> > > > > > > > >> On the other hand, I think improving the message error
>>>> with
>>>> > > > > > reference
>>>> > > > > > > to
>>>> > > > > > > > >> the metaspace option should help the most
>>>> > > > > > > > >> because user would not have to read all docs to fix it
>>>> > > > > > > > >> then maybe this change is not even needed.
>>>> > > > > > > > >>
>>>> > > > > > > > >> Best,
>>>> > > > > > > > >> Andrey
>>>> > > > > > > > >>
>>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>>>> sewen@apache.org>
>>>> > > > wrote:
>>>> > > > > > > > >>>
>>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
>>>> > > > > > > > >>>
>>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>>>> error
>>>> > on
>>>> > > > > > repeated
>>>> > > > > > > > >> job"
>>>> > > > > > > > >>> - I don't think we have consensus on the exact
>>>> solution,
>>>> > yet,
>>>> > > > and
>>>> > > > > > > some
>>>> > > > > > > > >> of
>>>> > > > > > > > >>> the changes might also have side effects that are
>>>> hard to
>>>> > > > > predict,
>>>> > > > > > > so I
>>>> > > > > > > > >> am
>>>> > > > > > > > >>> not sure we should rush this in.
>>>> > > > > > > > >>>
>>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>>>> > Metaspace"
>>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>>>> release?
>>>> > > It
>>>> > > > > > > changes
>>>> > > > > > > > >>> setups, takes away memory from heap / managed memory
>>>> on
>>>> > > > existing
>>>> > > > > > > setups
>>>> > > > > > > > >>> that keep their flink-conf.yaml.
>>>> > > > > > > > >>>
>>>> > > > > > > > >>> We may need to unblock the release form these two
>>>> issues
>>>> > and
>>>> > > > > think
>>>> > > > > > > > about
>>>> > > > > > > > >>> having 1.10.2 in the near future.
>>>> > > > > > > > >>>
>>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
>>>> carp84@gmail.com>
>>>> > > > wrote:
>>>> > > > > > > > >>>
>>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on
>>>> these
>>>> > two.
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>>> Best Regards,
>>>> > > > > > > > >>>> Yu
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>>>> imjark@gmail.com>
>>>> > > > wrote:
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
>>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>> From the Table SQL module, I think we should also
>>>> try to
>>>> > > get
>>>> > > > in
>>>> > > > > > the
>>>> > > > > > > > >>>>> following issues:
>>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
>>>> > > parameters
>>>> > > > > from
>>>> > > > > > > SQL
>>>> > > > > > > > >> CLI
>>>> > > > > > > > >>>>> environment
>>>> > > > > > > > >>>>> this allows users to set e.g. statebackend,
>>>> watermark
>>>> > > > interval,
>>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
>>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
>>>> > > > exception
>>>> > > > > is
>>>> > > > > > > > >> thrown
>>>> > > > > > > > >>>>> this will improve much experience when using SQL CLI
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>> Best,
>>>> > > > > > > > >>>>> Jark
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
>>>> carp84@gmail.com>
>>>> > > > wrote:
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added
>>>> 1.10.1 into
>>>> > > > > > > FLINK-16225
>>>> > > > > > > > >>>> fix
>>>> > > > > > > > >>>>>> versions and promoted its priority to Critical.
>>>> Will
>>>> > also
>>>> > > > > watch
>>>> > > > > > > the
>>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>> Best Regards,
>>>> > > > > > > > >>>>>> Yu
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
>>>> > > > > > > azagrebin@apache.org
>>>> > > > > > > > >
>>>> > > > > > > > >>>>>> wrote:
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>>> Hi Yu,
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
>>>> discussion!
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> Apart from
>>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
>>>> Metaspace
>>>> > to
>>>> > > > > > > minimise
>>>> > > > > > > > >>>> its
>>>> > > > > > > > >>>>>>> OutOfMemoryError
>>>> > > > > > > > >>>>>>> which should be merged soon
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> I think we should also try to get in the following
>>>> > > issues:
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be
>>>> > handled
>>>> > > > as
>>>> > > > > > > Fatal
>>>> > > > > > > > >>>>> Error
>>>> > > > > > > > >>>>>> in
>>>> > > > > > > > >>>>>>> TaskManager
>>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even in a
>>>> > better
>>>> > > > way
>>>> > > > > > > > because
>>>> > > > > > > > >>>>> OOM
>>>> > > > > > > > >>>>>>> failure should point users to the docs immediately
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
>>>> lifetime
>>>> > > of
>>>> > > > a
>>>> > > > > > slot
>>>> > > > > > > > >>>>>>> This should give a better protection against class
>>>> > > loading
>>>> > > > > > leaks
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
>>>> submitting
>>>> > > > batch
>>>> > > > > > job
>>>> > > > > > > > >>>>>> (instead
>>>> > > > > > > > >>>>>>> of AskTimeoutException)
>>>> > > > > > > > >>>>>>> This problem has recently happened for multiple
>>>> users
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> Best,
>>>> > > > > > > > >>>>>>> Andrey
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
>>>> > > > > > > > jingsonglee0@gmail.com>
>>>> > > > > > > > >>>>>>> wrote:
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the
>>>> 1.10.1
>>>> > > > release.
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>> Some issues are very important, Users are looking
>>>> > > forward
>>>> > > > to
>>>> > > > > > > them.
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>> Best,
>>>> > > > > > > > >>>>>>>> Jingsong Lee
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
>>>> > > > > > karmagyz@gmail.com>
>>>> > > > > > > > >>>>> wrote:
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>> Best,
>>>> > > > > > > > >>>>>>>>> Yangze Guo
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
>>>> > > > > > > > >>>>> tonysong820@gmail.com>
>>>> > > > > > > > >>>>>>>>> wrote:
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>> Yu,
>>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
>>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some
>>>> > inputs
>>>> > > > for
>>>> > > > > > > > >>>>>>> prioritizing
>>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to
>>>> me.
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>> Thank you~
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>> Xintong Song
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
>>>> > > > carp84@gmail.com>
>>>> > > > > > > > >>>> wrote:
>>>> > > > > > > > >>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
>>>> release
>>>> > of
>>>> > > a
>>>> > > > > > major
>>>> > > > > > > > >>>>>>> version
>>>> > > > > > > > >>>>>>>>> will
>>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
>>>> depending on
>>>> > > the
>>>> > > > > > speed
>>>> > > > > > > > >>>> of
>>>> > > > > > > > >>>>>>>> blocker
>>>> > > > > > > > >>>>>>>>>>> issue resolving:
>>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
>>>> > released
>>>> > > > on
>>>> > > > > > Jul
>>>> > > > > > > > >>>>> 3rd
>>>> > > > > > > > >>>>>>> [2]
>>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
>>>> > released
>>>> > > > on
>>>> > > > > > Oct
>>>> > > > > > > > >>>>> 19th
>>>> > > > > > > > >>>>>>> [4]
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of course,
>>>> but
>>>> > > could
>>>> > > > > use
>>>> > > > > > > > >>>> it
>>>> > > > > > > > >>>>>> as a
>>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me
>>>> know if
>>>> > any
>>>> > > > > > > > >>>> concerns
>>>> > > > > > > > >>>>>>>> Xintong.
>>>> > > > > > > > >>>>>>>>>>> Thanks.
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> Best Regards,
>>>> > > > > > > > >>>>>>>>>>> Yu
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> [1]
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>>>> > > > > > > > >>>>>>>>>>> [2]
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>>>> > > > > > > > >>>>>>>>>>> [3]
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>>>> > > > > > > > >>>>>>>>>>> [4]
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
>>>> > > > > > > > >>>>>> tonysong820@gmail.com>
>>>> > > > > > > > >>>>>>>>> wrote:
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering
>>>> to be
>>>> > > the
>>>> > > > > > > > >>>>> release
>>>> > > > > > > > >>>>>>>>> manager.
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
>>>> > > 1.10.1?
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> Thank you~
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> Xintong Song
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
>>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>> Best,
>>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > >
>>>> > ------------------------------------------------------------------
>>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will
>>>> also
>>>> > keep
>>>> > > > an
>>>> > > > > > > > >>>> eye
>>>> > > > > > > > >>>>> on
>>>> > > > > > > > >>>>>>>> these
>>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
>>>> > > > > > > > >>>>>>>>>>>>> Yu
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
>>>> > > > > > > > >>>>> hequn@apache.org>
>>>> > > > > > > > >>>>>>>>> wrote:
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
>>>> > > > volunteer
>>>> > > > > > > > >>>> as
>>>> > > > > > > > >>>>>> the
>>>> > > > > > > > >>>>>>>>> release
>>>> > > > > > > > >>>>>>>>>>>>>> manager!
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1]
>>>> which are
>>>> > > > > marked
>>>> > > > > > > > >>>>> as
>>>> > > > > > > > >>>>>> a
>>>> > > > > > > > >>>>>>>>> blocker:
>>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in
>>>> > NOTICE
>>>> > > > > files
>>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>>>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE
>>>> and
>>>> > > usrlib
>>>> > > > > > > > >>>>>> directory
>>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>>>> > > > > > > > >>>>> ClassNotFoundException
>>>> > > > > > > > >>>>>>> when
>>>> > > > > > > > >>>>>>>>> use
>>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
>>>> > > > submitting
>>>> > > > > > > > >>>>> batch
>>>> > > > > > > > >>>>>>> job
>>>> > > > > > > > >>>>>>>>>>>> (instead
>>>> > > > > > > > >>>>>>>>>>>>> of
>>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in
>>>> 1.10.1.
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> Best,
>>>> > > > > > > > >>>>>>>>>>>>>> Hequn
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> [1]
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > >
>>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>>>> > > > > > > > >>>> carp84@gmail.com>
>>>> > > > > > > > >>>>>>>> wrote:
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
>>>> Thanks a
>>>> > > lot!
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
>>>> > > > > > > > >>>>>>>>>>>>>>> Yu
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng
>>>> sun <
>>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I
>>>> would
>>>> > > > like
>>>> > > > > > > > >>>>> to
>>>> > > > > > > > >>>>>>> give
>>>> > > > > > > > >>>>>>>>> you a
>>>> > > > > > > > >>>>>>>>>>>>> hand
>>>> > > > > > > > >>>>>>>>>>>>>> at
>>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
>>>> finished.(If you
>>>> > > > > > > > >>>> need)
>>>> > > > > > > > >>>>>> :)
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
>>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
>>>> > > 下午5:49写道：
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
>>>> > released
>>>> > > > > > > > >>>>> Flink
>>>> > > > > > > > >>>>>>>>> 1.10.0. We
>>>> > > > > > > > >>>>>>>>>>>>>> already
>>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
>>>> improvements/bugs
>>>> > in
>>>> > > > > > > > >>>> the
>>>> > > > > > > > >>>>>>>>>>> release-1.10
>>>> > > > > > > > >>>>>>>>>>>>>>> branch,
>>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1
>>>> release
>>>> > > cycle.
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the
>>>> license and
>>>> > > > > > > > >>>> notice
>>>> > > > > > > > >>>>>>> file
>>>> > > > > > > > >>>>>>>> in
>>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
>>>> > > > > > > > >>>>>>>>>>>>>>>>> module
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak
>>>> in
>>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be
>>>> known
>>>> > > in
>>>> > > > > > > > >>>>>>>>> HiveCatalog
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
>>>> ArrayIndexOufOBoundsException in
>>>> > > > > > > > >>>>>>>> TypeExtractor
>>>> > > > > > > > >>>>>>>>>>> when
>>>> > > > > > > > >>>>>>>>>>>>>>> mapping
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is
>>>> failed if
>>>> > > > > > > > >>>> running
>>>> > > > > > > > >>>>>> in
>>>> > > > > > > > >>>>>>>>> blink
>>>> > > > > > > > >>>>>>>>>>>>> planner
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints
>>>> are not
>>>> > > > > > > > >>>> reset
>>>> > > > > > > > >>>>>> on
>>>> > > > > > > > >>>>>>>> task
>>>> > > > > > > > >>>>>>>>>>>>> recovery
>>>> > > > > > > > >>>>>>>>>>>>>> in
>>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function
>>>> using
>>>> > > sql
>>>> > > > > > > > >>>>>>> casuing
>>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation
>>>> failed
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following
>>>> issues
>>>> > > should
>>>> > > > > > > > >>>> be
>>>> > > > > > > > >>>>>>>> merged
>>>> > > > > > > > >>>>>>>>>>> before
>>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
>>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
>>>> issue):
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
>>>> Metaspace
>>>> > OOM
>>>> > > > > > > > >>>>> error
>>>> > > > > > > > >>>>>> on
>>>> > > > > > > > >>>>>>>>>>> repeated
>>>> > > > > > > > >>>>>>>>>>>>> job
>>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value
>>>> for JVM
>>>> > > > > > > > >>>>>> Metaspace
>>>> > > > > > > > >>>>>>> to
>>>> > > > > > > > >>>>>>>>>>>> minimise
>>>> > > > > > > > >>>>>>>>>>>>>> its
>>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces
>>>> wrong
>>>> > > > > > > > >>>>> aggregate
>>>> > > > > > > > >>>>>>>>> results
>>>> > > > > > > > >>>>>>>>>>> with
>>>> > > > > > > > >>>>>>>>>>>>>> state
>>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
>>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
>>>> extract
>>>> > > > > > > > >>>> correct
>>>> > > > > > > > >>>>>>>> unique
>>>> > > > > > > > >>>>>>>>> key
>>>> > > > > > > > >>>>>>>>>>>> for
>>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release
>>>> manager and
>>>> > > kick
>>>> > > > > > > > >>>>> off
>>>> > > > > > > > >>>>>>> the
>>>> > > > > > > > >>>>>>>>>>> release
>>>> > > > > > > > >>>>>>>>>>>>>>> process
>>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do
>>>> you
>>>> > > > > > > > >>>> think?
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
>>>> blocker
>>>> > > > > > > > >>>> issues
>>>> > > > > > > > >>>>>>> need
>>>> > > > > > > > >>>>>>>>> to be
>>>> > > > > > > > >>>>>>>>>>>>> fixed
>>>> > > > > > > > >>>>>>>>>>>>>> in
>>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
>>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
>>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>>>
>>>> > > > > > > > >>>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>> --
>>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
>>>> > > > > > > > >>>>>>>>
>>>> > > > > > > > >>>>>>>
>>>> > > > > > > > >>>>>>
>>>> > > > > > > > >>>>>
>>>> > > > > > > > >>>>
>>>> > > > > > > > >>
>>>> > > > > > > > >>
>>>> > > > > > > >
>>>> > > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Here comes the latest status of issues in 1.10.1 watch list:
* Blockers (4 left)
- [Under Discussion] FLINK-16018 Improve error reporting when submitting
batch job (instead of AskTimeoutException)
- [Closed] FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
job submission
- [Closed] FLINK-16170 SearchTemplateRequest ClassNotFoundException when
use flink-sql-connector-elasticsearch7
- [PR Approved] FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- [Closed] FLINK-16406 Increase default value for JVM Metaspace to
minimise its OutOfMemoryError
- [Closed] FLINK-16454 Update the copyright year in NOTICE files
- [In Progress] FLINK-16576 State inconsistency on restore with memory
state backends
- [New] [PR Submitted] FLINK-16705 LocalExecutor tears down MiniCluster
before client can retrieve JobResult
* Critical (2 left)
- [Closed] FLINK-16047 Blink planner produces wrong aggregate results
with state clean up
- [PR Submitted] FLINK-16070 Blink planner can not extract correct unique
key for UpsertStreamTableSink
- [Fix for 1.10.1 Done] FLINK-16225 Metaspace Out Of Memory should be
handled as Fatal Error in TaskManager
- [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
There's a new blocker added. Please let me know if you find any other
issues should be put into the watch list. Thanks.
Best Regards,
Yu
On Sat, 21 Mar 2020 at 14:37, Yu Li <ca...@gmail.com> wrote:
> Hi All,
>
> Here is the status update of issues in 1.10.1 watch list:
>
> * Blockers (7)
>
> - [Under Discussion] FLINK-16018 Improve error reporting when submitting
> batch job (instead of AskTimeoutException)
>
> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
> on repeated job submission
>
> - [PR Submitted] FLINK-16170 SearchTemplateRequest
> ClassNotFoundException when use flink-sql-connector-elasticsearch7
>
> - [PR Submitted] FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>
> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> minimise its OutOfMemoryError
>
> - [Closed] FLINK-16454 Update the copyright year in NOTICE files
>
> - [Open] FLINK-16576 State inconsistency on restore with memory state
> backends
>
>
> * Critical (4)
>
> - [Closed] FLINK-16047 Blink planner produces wrong aggregate results
> with state clean up
>
> - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> unique key for UpsertStreamTableSink
>
> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> handled as Fatal Error in TaskManager
>
> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
>
> Please let me know if you see any new blockers to add to the list. Thanks.
>
> Best Regards,
> Yu
>
>
> On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
>
>> Thanks for the updates Till!
>>
>> For FLINK-16018, maybe we could create two sub-tasks for easy and
>> complete fix separately, and only include the easy one in 1.10.1? Or please
>> just feel free to postpone the whole task to 1.10.2 if "all or nothing"
>> policy is preferred (smile). Thanks.
>>
>> Best Regards,
>> Yu
>>
>>
>> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org> wrote:
>>
>>> +1 for a soonish bug fix release. Thanks for volunteering as our release
>>> manager Yu.
>>>
>>> I think we can soon merge the increase of metaspace size and improving
>>> the
>>> error message. The assumption is that we currently don't have too many
>>> small Flink 1.10 deployments with a process size <= 1GB. Of course, the
>>> sooner we release the bug fix release, the fewer deployments will be
>>> affected by this change.
>>>
>>> For FLINK-16018, I think there would be an easy solution which we could
>>> include in the bug fix release. The proper fix will most likely take a
>>> bit
>>> longer, though.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <az...@apache.org>
>>> wrote:
>>>
>>> > > @Andrey and @Xintong - could we have a quick poll on the user mailing
>>> > list
>>> > > about increasing the metaspace size in Flink 1.10.1? Specifically
>>> asking
>>> > > for who has very small TM setups?
>>> >
>>> > There has been a survey about this topic since 10 days:
>>> >
>>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
>>> > I can ask there about the small TM setups.
>>> >
>>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
>>> >
>>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on
>>> restore
>>> > with
>>> > > memory state backends
>>> > >
>>> > > Let me recompile the watching list with recent feedbacks. There're
>>> > totally
>>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
>>> > > already resolved and 31 left, and the below ones are watched (meaning
>>> > that
>>> > > once the below ones got fixed, we will kick of the releasing with
>>> left
>>> > ones
>>> > > postponed to 1.10.2 unless objections):
>>> > >
>>> > > * Blockers (7)
>>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
>>> > submitting
>>> > > batch job (instead of AskTimeoutException)
>>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
>>> error
>>> > > on repeated job submission
>>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
>>> > ClassNotFoundException
>>> > > when use flink-sql-connector-elasticsearch7
>>> > > - [PR Submitted] FLINK-16262 Class loader problem with
>>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>>> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>>> > > minimise its OutOfMemoryError
>>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
>>> > > - [Open] FLINK-16576 State inconsistency on restore with memory
>>> state
>>> > > backends
>>> > >
>>> > > * Critical (4)
>>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate results
>>> > with
>>> > > state clean up
>>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>>> > unique
>>> > > key for UpsertStreamTableSink
>>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
>>> > > handled as Fatal Error in TaskManager
>>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
>>> slot
>>> > >
>>> > > Please raise your hand if you find any other issues should be put
>>> into
>>> > this
>>> > > list, thanks.
>>> > >
>>> > > Please also note that 1.10.2 version is already created (thanks for
>>> the
>>> > > help @jincheng), and feel free to adjust/assign fix version to it
>>> when
>>> > > necessary.
>>> > >
>>> > > Best Regards,
>>> > > Yu
>>> > >
>>> > >
>>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
>>> > >
>>> > > > @Andrey and @Xintong - could we have a quick poll on the user
>>> mailing
>>> > > list
>>> > > > about increasing the metaspace size in Flink 1.10.1? Specifically
>>> > asking
>>> > > > for who has very small TM setups?
>>> > > >
>>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
>>> > > >
>>> > > > > Thanks for the reminder Stephan and the inputs/discussion Andrey
>>> and
>>> > > > > Xintong. I totally agree that we should try best not to
>>> introduce new
>>> > > > > issues when resolving the existing one. Will watch closely about
>>> the
>>> > > > > discussion and conclusion around metaspace configuration.
>>> > > > >
>>> > > > > From the point as a developer instead of RM, I'm optimistic that
>>> > > > reserving
>>> > > > > 160MB more memory to JVM metaspace by default won't cause a big
>>> > > problem,
>>> > > > > based on the assumption that nowadays assigning 2GB/4GB memory
>>> per TM
>>> > > is
>>> > > > > the common case.
>>> > > > >
>>> > > > > Best Regards,
>>> > > > > Yu
>>> > > > >
>>> > > > >
>>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <
>>> tonysong820@gmail.com>
>>> > > > wrote:
>>> > > > >
>>> > > > > > I think in FLINK-16406 we have not only increased the default
>>> value
>>> > > for
>>> > > > > > metaspace, but also increased the 'process.size' in default
>>> > > > > > 'flink-conf.yaml'.
>>> > > > > >
>>> > > > > > That means:
>>> > > > > >
>>> > > > > > - For people using the default 'flink-conf.yaml', it's not
>>> > really
>>> > > > > taking
>>> > > > > > memory from heap / managed for metaspace.
>>> > > > > > - For people using custom configuration files:
>>> > > > > > - If they are specifying 'flink.size', then metaspace
>>> does
>>> > not
>>> > > > > really
>>> > > > > > affect the flink memory internal components (heap,
>>> managed,
>>> > > > etc.).
>>> > > > > > - If they are specifying 'process.size' (most likely
>>> active
>>> > > > > setups),
>>> > > > > > then they will have less heap / managed memory, or worse,
>>> > their
>>> > > > > > configuration may not work without adjustment.
>>> > > > > > - For a TM with large process memory (e.g., 4 GB or
>>> more),
>>> > > it
>>> > > > is
>>> > > > > > probably ok that the metaspace takes 160 MB more
>>> memory.
>>> > > > > > - For a TM with smaller total memory (1 or 2 GB) the
>>> > > metaspace
>>> > > > > > increase is more likely to cause problem.
>>> > > > > >
>>> > > > > > So basically only people have small 'process.size' in custom
>>> config
>>> > > > file
>>> > > > > > are really affected. I'm not sure what is the proportion of
>>> such
>>> > use
>>> > > > > cases
>>> > > > > > though. (From questions asked on the user ML, probably not
>>> much).
>>> > > > > >
>>> > > > > > Thank you~
>>> > > > > >
>>> > > > > > Xintong Song
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <
>>> sewen@apache.org>
>>> > > > wrote:
>>> > > > > >
>>> > > > > > > No need to revert it now - I am not saying it should not go
>>> into
>>> > > > > 1.10.1,
>>> > > > > > I
>>> > > > > > > am just saying this is not clear to me yet.
>>> > > > > > >
>>> > > > > > > We have to trade off the fact that we may break some
>>> deployments
>>> > > with
>>> > > > > the
>>> > > > > > > fact that we will "safe" a lot of new deployments.
>>> > > > > > > I simply lack the data points / insight at the moment to
>>> > understand
>>> > > > how
>>> > > > > > > significant both cases are, meaning how many users would be
>>> > > affected
>>> > > > > and
>>> > > > > > > how badly.
>>> > > > > > >
>>> > > > > > > Independent of that, improving the error message is always
>>> > helpful.
>>> > > > > > >
>>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>>> > > > > > > azagrebin.apache@gmail.com>
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>>> expect
>>> > > to
>>> > > > > > > upgrade
>>> > > > > > > > > that without config adjustments. That might not be
>>> possible
>>> > > with
>>> > > > > that
>>> > > > > > > > > change.
>>> > > > > > > >
>>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
>>> > > improve
>>> > > > > > error
>>> > > > > > > > message and docs.
>>> > > > > > > >
>>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <sewen@apache.org
>>> >
>>> > > wrote:
>>> > > > > > > > >
>>> > > > > > > > > @Andrey about the increase in metaspace size
>>> > > > > > > > > - I have no concerns for 1.11.0.
>>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>>> expect
>>> > > to
>>> > > > > > > upgrade
>>> > > > > > > > > that without config adjustments. That might not be
>>> possible
>>> > > with
>>> > > > > that
>>> > > > > > > > > change.
>>> > > > > > > > >
>>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
>>> > > > > > > > azagrebin.apache@gmail.com>
>>> > > > > > > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > >>
>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>>> error
>>> > on
>>> > > > > > repeated
>>> > > > > > > > >> job”
>>> > > > > > > > >>
>>> > > > > > > > >> My understanding that the issue is basically covered by:
>>> > > > > > > > >>
>>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
>>> handled as
>>> > > > Fatal
>>> > > > > > > Error
>>> > > > > > > > >> in TaskManager
>>> > > > > > > > >> no full consensus there but improving error message
>>> for
>>> > > > existing
>>> > > > > > > task
>>> > > > > > > > >> thread fatal handling could be done at least
>>> > > > > > > > >>
>>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM
>>> Metaspace to
>>> > > > > minimise
>>> > > > > > > its
>>> > > > > > > > >> OutOfMemoryError
>>> > > > > > > > >> see further
>>> > > > > > > > >>
>>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
>>> > > > dynamically
>>> > > > > > > > loaded
>>> > > > > > > > >> AWS connectors
>>> > > > > > > > >> not sure whether this is a blocker but looks close to
>>> be
>>> > > > resolved
>>> > > > > > > > >>
>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>>> > Metaspace"
>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>>> release?
>>> > > It
>>> > > > > > > changes
>>> > > > > > > > >>> setups, takes away memory from heap / managed memory on
>>> > > > existing
>>> > > > > > > setups
>>> > > > > > > > >>> that keep their flink-conf.yaml.
>>> > > > > > > > >>
>>> > > > > > > > >> My understanding was that increasing to 256m resolved
>>> the
>>> > > > reported
>>> > > > > > > > problems
>>> > > > > > > > >> and we decided to make the change so I have merged it
>>> today
>>> > as
>>> > > > > there
>>> > > > > > > > were
>>> > > > > > > > >> no more concerns.
>>> > > > > > > > >> If there are concerns I can revert it.
>>> > > > > > > > >>
>>> > > > > > > > >> On the other hand, I think improving the message error
>>> with
>>> > > > > > reference
>>> > > > > > > to
>>> > > > > > > > >> the metaspace option should help the most
>>> > > > > > > > >> because user would not have to read all docs to fix it
>>> > > > > > > > >> then maybe this change is not even needed.
>>> > > > > > > > >>
>>> > > > > > > > >> Best,
>>> > > > > > > > >> Andrey
>>> > > > > > > > >>
>>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>>> sewen@apache.org>
>>> > > > wrote:
>>> > > > > > > > >>>
>>> > > > > > > > >>> Good idea to go ahead with 1.10.1
>>> > > > > > > > >>>
>>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>>> error
>>> > on
>>> > > > > > repeated
>>> > > > > > > > >> job"
>>> > > > > > > > >>> - I don't think we have consensus on the exact
>>> solution,
>>> > yet,
>>> > > > and
>>> > > > > > > some
>>> > > > > > > > >> of
>>> > > > > > > > >>> the changes might also have side effects that are hard
>>> to
>>> > > > > predict,
>>> > > > > > > so I
>>> > > > > > > > >> am
>>> > > > > > > > >>> not sure we should rush this in.
>>> > > > > > > > >>>
>>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>>> > Metaspace"
>>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>>> release?
>>> > > It
>>> > > > > > > changes
>>> > > > > > > > >>> setups, takes away memory from heap / managed memory on
>>> > > > existing
>>> > > > > > > setups
>>> > > > > > > > >>> that keep their flink-conf.yaml.
>>> > > > > > > > >>>
>>> > > > > > > > >>> We may need to unblock the release form these two
>>> issues
>>> > and
>>> > > > > think
>>> > > > > > > > about
>>> > > > > > > > >>> having 1.10.2 in the near future.
>>> > > > > > > > >>>
>>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <
>>> carp84@gmail.com>
>>> > > > wrote:
>>> > > > > > > > >>>
>>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on
>>> these
>>> > two.
>>> > > > > > > > >>>>
>>> > > > > > > > >>>> Best Regards,
>>> > > > > > > > >>>> Yu
>>> > > > > > > > >>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>>> imjark@gmail.com>
>>> > > > wrote:
>>> > > > > > > > >>>>
>>> > > > > > > > >>>>> Thanks for driving this release, Yu!
>>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>> From the Table SQL module, I think we should also
>>> try to
>>> > > get
>>> > > > in
>>> > > > > > the
>>> > > > > > > > >>>>> following issues:
>>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
>>> > > parameters
>>> > > > > from
>>> > > > > > > SQL
>>> > > > > > > > >> CLI
>>> > > > > > > > >>>>> environment
>>> > > > > > > > >>>>> this allows users to set e.g. statebackend, watermark
>>> > > > interval,
>>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
>>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
>>> > > > exception
>>> > > > > is
>>> > > > > > > > >> thrown
>>> > > > > > > > >>>>> this will improve much experience when using SQL CLI
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>> Best,
>>> > > > > > > > >>>>> Jark
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <
>>> carp84@gmail.com>
>>> > > > wrote:
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1
>>> into
>>> > > > > > > FLINK-16225
>>> > > > > > > > >>>> fix
>>> > > > > > > > >>>>>> versions and promoted its priority to Critical. Will
>>> > also
>>> > > > > watch
>>> > > > > > > the
>>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>> Best Regards,
>>> > > > > > > > >>>>>> Yu
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
>>> > > > > > > azagrebin@apache.org
>>> > > > > > > > >
>>> > > > > > > > >>>>>> wrote:
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>>> Hi Yu,
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
>>> discussion!
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> Apart from
>>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
>>> Metaspace
>>> > to
>>> > > > > > > minimise
>>> > > > > > > > >>>> its
>>> > > > > > > > >>>>>>> OutOfMemoryError
>>> > > > > > > > >>>>>>> which should be merged soon
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> I think we should also try to get in the following
>>> > > issues:
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be
>>> > handled
>>> > > > as
>>> > > > > > > Fatal
>>> > > > > > > > >>>>> Error
>>> > > > > > > > >>>>>> in
>>> > > > > > > > >>>>>>> TaskManager
>>> > > > > > > > >>>>>>> This should solve the Metaspace problem even in a
>>> > better
>>> > > > way
>>> > > > > > > > because
>>> > > > > > > > >>>>> OOM
>>> > > > > > > > >>>>>>> failure should point users to the docs immediately
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
>>> lifetime
>>> > > of
>>> > > > a
>>> > > > > > slot
>>> > > > > > > > >>>>>>> This should give a better protection against class
>>> > > loading
>>> > > > > > leaks
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
>>> submitting
>>> > > > batch
>>> > > > > > job
>>> > > > > > > > >>>>>> (instead
>>> > > > > > > > >>>>>>> of AskTimeoutException)
>>> > > > > > > > >>>>>>> This problem has recently happened for multiple
>>> users
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> Best,
>>> > > > > > > > >>>>>>> Andrey
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
>>> > > > > > > > jingsonglee0@gmail.com>
>>> > > > > > > > >>>>>>> wrote:
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
>>> > > > release.
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>> Some issues are very important, Users are looking
>>> > > forward
>>> > > > to
>>> > > > > > > them.
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>> Best,
>>> > > > > > > > >>>>>>>> Jingsong Lee
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
>>> > > > > > karmagyz@gmail.com>
>>> > > > > > > > >>>>> wrote:
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>> Best,
>>> > > > > > > > >>>>>>>>> Yangze Guo
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
>>> > > > > > > > >>>>> tonysong820@gmail.com>
>>> > > > > > > > >>>>>>>>> wrote:
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>> Yu,
>>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
>>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some
>>> > inputs
>>> > > > for
>>> > > > > > > > >>>>>>> prioritizing
>>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>> Thank you~
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>> Xintong Song
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
>>> > > > carp84@gmail.com>
>>> > > > > > > > >>>> wrote:
>>> > > > > > > > >>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
>>> release
>>> > of
>>> > > a
>>> > > > > > major
>>> > > > > > > > >>>>>>> version
>>> > > > > > > > >>>>>>>>> will
>>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started,
>>> depending on
>>> > > the
>>> > > > > > speed
>>> > > > > > > > >>>> of
>>> > > > > > > > >>>>>>>> blocker
>>> > > > > > > > >>>>>>>>>>> issue resolving:
>>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
>>> > released
>>> > > > on
>>> > > > > > Jul
>>> > > > > > > > >>>>> 3rd
>>> > > > > > > > >>>>>>> [2]
>>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
>>> > released
>>> > > > on
>>> > > > > > Oct
>>> > > > > > > > >>>>> 19th
>>> > > > > > > > >>>>>>> [4]
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of course,
>>> but
>>> > > could
>>> > > > > use
>>> > > > > > > > >>>> it
>>> > > > > > > > >>>>>> as a
>>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me know
>>> if
>>> > any
>>> > > > > > > > >>>> concerns
>>> > > > > > > > >>>>>>>> Xintong.
>>> > > > > > > > >>>>>>>>>>> Thanks.
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> Best Regards,
>>> > > > > > > > >>>>>>>>>>> Yu
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> [1]
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>>> > > > > > > > >>>>>>>>>>> [2]
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>>> > > > > > > > >>>>>>>>>>> [3]
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>>> > > > > > > > >>>>>>>>>>> [4]
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
>>> > > > > > > > >>>>>> tonysong820@gmail.com>
>>> > > > > > > > >>>>>>>>> wrote:
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering
>>> to be
>>> > > the
>>> > > > > > > > >>>>> release
>>> > > > > > > > >>>>>>>>> manager.
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
>>> > > 1.10.1?
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> Thank you~
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> Xintong Song
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
>>> > > > > > > > >>>>>>>>>>>>> +1 on my side
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>> Best,
>>> > > > > > > > >>>>>>>>>>>>> Zhijiang
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > >
>>> > ------------------------------------------------------------------
>>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will
>>> also
>>> > keep
>>> > > > an
>>> > > > > > > > >>>> eye
>>> > > > > > > > >>>>> on
>>> > > > > > > > >>>>>>>> these
>>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>> Best Regards,
>>> > > > > > > > >>>>>>>>>>>>> Yu
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
>>> > > > > > > > >>>>> hequn@apache.org>
>>> > > > > > > > >>>>>>>>> wrote:
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
>>> > > > volunteer
>>> > > > > > > > >>>> as
>>> > > > > > > > >>>>>> the
>>> > > > > > > > >>>>>>>>> release
>>> > > > > > > > >>>>>>>>>>>>>> manager!
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1]
>>> which are
>>> > > > > marked
>>> > > > > > > > >>>>> as
>>> > > > > > > > >>>>>> a
>>> > > > > > > > >>>>>>>>> blocker:
>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in
>>> > NOTICE
>>> > > > > files
>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
>>> > > usrlib
>>> > > > > > > > >>>>>> directory
>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>>> > > > > > > > >>>>> ClassNotFoundException
>>> > > > > > > > >>>>>>> when
>>> > > > > > > > >>>>>>>>> use
>>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
>>> > > > submitting
>>> > > > > > > > >>>>> batch
>>> > > > > > > > >>>>>>> job
>>> > > > > > > > >>>>>>>>>>>> (instead
>>> > > > > > > > >>>>>>>>>>>>> of
>>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in
>>> 1.10.1.
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> Best,
>>> > > > > > > > >>>>>>>>>>>>>> Hequn
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> [1]
>>> > > > > > > > >>>>>>>>>
>>> > > > > > >
>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>>> > > > > > > > >>>> carp84@gmail.com>
>>> > > > > > > > >>>>>>>> wrote:
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
>>> Thanks a
>>> > > lot!
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
>>> > > > > > > > >>>>>>>>>>>>>>> Yu
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun
>>> <
>>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>> wrote:
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I
>>> would
>>> > > > like
>>> > > > > > > > >>>>> to
>>> > > > > > > > >>>>>>> give
>>> > > > > > > > >>>>>>>>> you a
>>> > > > > > > > >>>>>>>>>>>>> hand
>>> > > > > > > > >>>>>>>>>>>>>> at
>>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is
>>> finished.(If you
>>> > > > > > > > >>>> need)
>>> > > > > > > > >>>>>> :)
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>> Best,
>>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
>>> > > 下午5:49写道：
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
>>> > released
>>> > > > > > > > >>>>> Flink
>>> > > > > > > > >>>>>>>>> 1.10.0. We
>>> > > > > > > > >>>>>>>>>>>>>> already
>>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
>>> improvements/bugs
>>> > in
>>> > > > > > > > >>>> the
>>> > > > > > > > >>>>>>>>>>> release-1.10
>>> > > > > > > > >>>>>>>>>>>>>>> branch,
>>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release
>>> > > cycle.
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license
>>> and
>>> > > > > > > > >>>> notice
>>> > > > > > > > >>>>>>> file
>>> > > > > > > > >>>>>>>> in
>>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
>>> > > > > > > > >>>>>>>>>>>>>>>>> module
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak
>>> in
>>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be
>>> known
>>> > > in
>>> > > > > > > > >>>>>>>>> HiveCatalog
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
>>> ArrayIndexOufOBoundsException in
>>> > > > > > > > >>>>>>>> TypeExtractor
>>> > > > > > > > >>>>>>>>>>> when
>>> > > > > > > > >>>>>>>>>>>>>>> mapping
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed
>>> if
>>> > > > > > > > >>>> running
>>> > > > > > > > >>>>>> in
>>> > > > > > > > >>>>>>>>> blink
>>> > > > > > > > >>>>>>>>>>>>> planner
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints
>>> are not
>>> > > > > > > > >>>> reset
>>> > > > > > > > >>>>>> on
>>> > > > > > > > >>>>>>>> task
>>> > > > > > > > >>>>>>>>>>>>> recovery
>>> > > > > > > > >>>>>>>>>>>>>> in
>>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function
>>> using
>>> > > sql
>>> > > > > > > > >>>>>>> casuing
>>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation
>>> failed
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues
>>> > > should
>>> > > > > > > > >>>> be
>>> > > > > > > > >>>>>>>> merged
>>> > > > > > > > >>>>>>>>>>> before
>>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
>>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
>>> issue):
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes
>>> Metaspace
>>> > OOM
>>> > > > > > > > >>>>> error
>>> > > > > > > > >>>>>> on
>>> > > > > > > > >>>>>>>>>>> repeated
>>> > > > > > > > >>>>>>>>>>>>> job
>>> > > > > > > > >>>>>>>>>>>>>>>>> submission
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
>>> JVM
>>> > > > > > > > >>>>>> Metaspace
>>> > > > > > > > >>>>>>> to
>>> > > > > > > > >>>>>>>>>>>> minimise
>>> > > > > > > > >>>>>>>>>>>>>> its
>>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces
>>> wrong
>>> > > > > > > > >>>>> aggregate
>>> > > > > > > > >>>>>>>>> results
>>> > > > > > > > >>>>>>>>>>> with
>>> > > > > > > > >>>>>>>>>>>>>> state
>>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
>>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
>>> extract
>>> > > > > > > > >>>> correct
>>> > > > > > > > >>>>>>>> unique
>>> > > > > > > > >>>>>>>>> key
>>> > > > > > > > >>>>>>>>>>>> for
>>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager
>>> and
>>> > > kick
>>> > > > > > > > >>>>> off
>>> > > > > > > > >>>>>>> the
>>> > > > > > > > >>>>>>>>>>> release
>>> > > > > > > > >>>>>>>>>>>>>>> process
>>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do
>>> you
>>> > > > > > > > >>>> think?
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
>>> blocker
>>> > > > > > > > >>>> issues
>>> > > > > > > > >>>>>>> need
>>> > > > > > > > >>>>>>>>> to be
>>> > > > > > > > >>>>>>>>>>>>> fixed
>>> > > > > > > > >>>>>>>>>>>>>> in
>>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
>>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
>>> > > > > > > > >>>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>>>
>>> > > > > > > > >>>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>> --
>>> > > > > > > > >>>>>>>> Best, Jingsong Lee
>>> > > > > > > > >>>>>>>>
>>> > > > > > > > >>>>>>>
>>> > > > > > > > >>>>>>
>>> > > > > > > > >>>>>
>>> > > > > > > > >>>>
>>> > > > > > > > >>
>>> > > > > > > > >>
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Hi All,
Here is the status update of issues in 1.10.1 watch list:
* Blockers (7)
- [Under Discussion] FLINK-16018 Improve error reporting when submitting
batch job (instead of AskTimeoutException)
- [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
on repeated job submission
- [PR Submitted] FLINK-16170 SearchTemplateRequest ClassNotFoundException
when use flink-sql-connector-elasticsearch7
- [PR Submitted] FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- [Closed] FLINK-16406 Increase default value for JVM Metaspace to
minimise its OutOfMemoryError
- [Closed] FLINK-16454 Update the copyright year in NOTICE files
- [Open] FLINK-16576 State inconsistency on restore with memory state
backends
* Critical (4)
- [Closed] FLINK-16047 Blink planner produces wrong aggregate results
with state clean up
- [PR Submitted] FLINK-16070 Blink planner can not extract correct unique
key for UpsertStreamTableSink
- [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
handled as Fatal Error in TaskManager
- [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
Please let me know if you see any new blockers to add to the list. Thanks.
Best Regards,
Yu
On Wed, 18 Mar 2020 at 00:11, Yu Li <ca...@gmail.com> wrote:
> Thanks for the updates Till!
>
> For FLINK-16018, maybe we could create two sub-tasks for easy and complete
> fix separately, and only include the easy one in 1.10.1? Or please just
> feel free to postpone the whole task to 1.10.2 if "all or nothing" policy
> is preferred (smile). Thanks.
>
> Best Regards,
> Yu
>
>
> On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org> wrote:
>
>> +1 for a soonish bug fix release. Thanks for volunteering as our release
>> manager Yu.
>>
>> I think we can soon merge the increase of metaspace size and improving the
>> error message. The assumption is that we currently don't have too many
>> small Flink 1.10 deployments with a process size <= 1GB. Of course, the
>> sooner we release the bug fix release, the fewer deployments will be
>> affected by this change.
>>
>> For FLINK-16018, I think there would be an easy solution which we could
>> include in the bug fix release. The proper fix will most likely take a bit
>> longer, though.
>>
>> Cheers,
>> Till
>>
>> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <az...@apache.org>
>> wrote:
>>
>> > > @Andrey and @Xintong - could we have a quick poll on the user mailing
>> > list
>> > > about increasing the metaspace size in Flink 1.10.1? Specifically
>> asking
>> > > for who has very small TM setups?
>> >
>> > There has been a survey about this topic since 10 days:
>> >
>> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
>> > I can ask there about the small TM setups.
>> >
>> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
>> >
>> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore
>> > with
>> > > memory state backends
>> > >
>> > > Let me recompile the watching list with recent feedbacks. There're
>> > totally
>> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
>> > > already resolved and 31 left, and the below ones are watched (meaning
>> > that
>> > > once the below ones got fixed, we will kick of the releasing with left
>> > ones
>> > > postponed to 1.10.2 unless objections):
>> > >
>> > > * Blockers (7)
>> > > - [Under Discussion] FLINK-16018 Improve error reporting when
>> > submitting
>> > > batch job (instead of AskTimeoutException)
>> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
>> error
>> > > on repeated job submission
>> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
>> > ClassNotFoundException
>> > > when use flink-sql-connector-elasticsearch7
>> > > - [PR Submitted] FLINK-16262 Class loader problem with
>> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
>> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
>> > > minimise its OutOfMemoryError
>> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
>> > > - [Open] FLINK-16576 State inconsistency on restore with memory
>> state
>> > > backends
>> > >
>> > > * Critical (4)
>> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate results
>> > with
>> > > state clean up
>> > > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
>> > unique
>> > > key for UpsertStreamTableSink
>> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
>> > > handled as Fatal Error in TaskManager
>> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
>> slot
>> > >
>> > > Please raise your hand if you find any other issues should be put into
>> > this
>> > > list, thanks.
>> > >
>> > > Please also note that 1.10.2 version is already created (thanks for
>> the
>> > > help @jincheng), and feel free to adjust/assign fix version to it when
>> > > necessary.
>> > >
>> > > Best Regards,
>> > > Yu
>> > >
>> > >
>> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
>> > >
>> > > > @Andrey and @Xintong - could we have a quick poll on the user
>> mailing
>> > > list
>> > > > about increasing the metaspace size in Flink 1.10.1? Specifically
>> > asking
>> > > > for who has very small TM setups?
>> > > >
>> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
>> > > >
>> > > > > Thanks for the reminder Stephan and the inputs/discussion Andrey
>> and
>> > > > > Xintong. I totally agree that we should try best not to introduce
>> new
>> > > > > issues when resolving the existing one. Will watch closely about
>> the
>> > > > > discussion and conclusion around metaspace configuration.
>> > > > >
>> > > > > From the point as a developer instead of RM, I'm optimistic that
>> > > > reserving
>> > > > > 160MB more memory to JVM metaspace by default won't cause a big
>> > > problem,
>> > > > > based on the assumption that nowadays assigning 2GB/4GB memory
>> per TM
>> > > is
>> > > > > the common case.
>> > > > >
>> > > > > Best Regards,
>> > > > > Yu
>> > > > >
>> > > > >
>> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <tonysong820@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > I think in FLINK-16406 we have not only increased the default
>> value
>> > > for
>> > > > > > metaspace, but also increased the 'process.size' in default
>> > > > > > 'flink-conf.yaml'.
>> > > > > >
>> > > > > > That means:
>> > > > > >
>> > > > > > - For people using the default 'flink-conf.yaml', it's not
>> > really
>> > > > > taking
>> > > > > > memory from heap / managed for metaspace.
>> > > > > > - For people using custom configuration files:
>> > > > > > - If they are specifying 'flink.size', then metaspace does
>> > not
>> > > > > really
>> > > > > > affect the flink memory internal components (heap,
>> managed,
>> > > > etc.).
>> > > > > > - If they are specifying 'process.size' (most likely
>> active
>> > > > > setups),
>> > > > > > then they will have less heap / managed memory, or worse,
>> > their
>> > > > > > configuration may not work without adjustment.
>> > > > > > - For a TM with large process memory (e.g., 4 GB or
>> more),
>> > > it
>> > > > is
>> > > > > > probably ok that the metaspace takes 160 MB more
>> memory.
>> > > > > > - For a TM with smaller total memory (1 or 2 GB) the
>> > > metaspace
>> > > > > > increase is more likely to cause problem.
>> > > > > >
>> > > > > > So basically only people have small 'process.size' in custom
>> config
>> > > > file
>> > > > > > are really affected. I'm not sure what is the proportion of such
>> > use
>> > > > > cases
>> > > > > > though. (From questions asked on the user ML, probably not
>> much).
>> > > > > >
>> > > > > > Thank you~
>> > > > > >
>> > > > > > Xintong Song
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <sewen@apache.org
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > > No need to revert it now - I am not saying it should not go
>> into
>> > > > > 1.10.1,
>> > > > > > I
>> > > > > > > am just saying this is not clear to me yet.
>> > > > > > >
>> > > > > > > We have to trade off the fact that we may break some
>> deployments
>> > > with
>> > > > > the
>> > > > > > > fact that we will "safe" a lot of new deployments.
>> > > > > > > I simply lack the data points / insight at the moment to
>> > understand
>> > > > how
>> > > > > > > significant both cases are, meaning how many users would be
>> > > affected
>> > > > > and
>> > > > > > > how badly.
>> > > > > > >
>> > > > > > > Independent of that, improving the error message is always
>> > helpful.
>> > > > > > >
>> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
>> > > > > > > azagrebin.apache@gmail.com>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>> expect
>> > > to
>> > > > > > > upgrade
>> > > > > > > > > that without config adjustments. That might not be
>> possible
>> > > with
>> > > > > that
>> > > > > > > > > change.
>> > > > > > > >
>> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
>> > > improve
>> > > > > > error
>> > > > > > > > message and docs.
>> > > > > > > >
>> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org>
>> > > wrote:
>> > > > > > > > >
>> > > > > > > > > @Andrey about the increase in metaspace size
>> > > > > > > > > - I have no concerns for 1.11.0.
>> > > > > > > > > - For 1.10.1 I am not completely sure, because users
>> expect
>> > > to
>> > > > > > > upgrade
>> > > > > > > > > that without config adjustments. That might not be
>> possible
>> > > with
>> > > > > that
>> > > > > > > > > change.
>> > > > > > > > >
>> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
>> > > > > > > > azagrebin.apache@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >>
>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>> error
>> > on
>> > > > > > repeated
>> > > > > > > > >> job”
>> > > > > > > > >>
>> > > > > > > > >> My understanding that the issue is basically covered by:
>> > > > > > > > >>
>> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be
>> handled as
>> > > > Fatal
>> > > > > > > Error
>> > > > > > > > >> in TaskManager
>> > > > > > > > >> no full consensus there but improving error message for
>> > > > existing
>> > > > > > > task
>> > > > > > > > >> thread fatal handling could be done at least
>> > > > > > > > >>
>> > > > > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace
>> to
>> > > > > minimise
>> > > > > > > its
>> > > > > > > > >> OutOfMemoryError
>> > > > > > > > >> see further
>> > > > > > > > >>
>> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
>> > > > dynamically
>> > > > > > > > loaded
>> > > > > > > > >> AWS connectors
>> > > > > > > > >> not sure whether this is a blocker but looks close to be
>> > > > resolved
>> > > > > > > > >>
>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>> > Metaspace"
>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>> release?
>> > > It
>> > > > > > > changes
>> > > > > > > > >>> setups, takes away memory from heap / managed memory on
>> > > > existing
>> > > > > > > setups
>> > > > > > > > >>> that keep their flink-conf.yaml.
>> > > > > > > > >>
>> > > > > > > > >> My understanding was that increasing to 256m resolved the
>> > > > reported
>> > > > > > > > problems
>> > > > > > > > >> and we decided to make the change so I have merged it
>> today
>> > as
>> > > > > there
>> > > > > > > > were
>> > > > > > > > >> no more concerns.
>> > > > > > > > >> If there are concerns I can revert it.
>> > > > > > > > >>
>> > > > > > > > >> On the other hand, I think improving the message error
>> with
>> > > > > > reference
>> > > > > > > to
>> > > > > > > > >> the metaspace option should help the most
>> > > > > > > > >> because user would not have to read all docs to fix it
>> > > > > > > > >> then maybe this change is not even needed.
>> > > > > > > > >>
>> > > > > > > > >> Best,
>> > > > > > > > >> Andrey
>> > > > > > > > >>
>> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <
>> sewen@apache.org>
>> > > > wrote:
>> > > > > > > > >>>
>> > > > > > > > >>> Good idea to go ahead with 1.10.1
>> > > > > > > > >>>
>> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM
>> error
>> > on
>> > > > > > repeated
>> > > > > > > > >> job"
>> > > > > > > > >>> - I don't think we have consensus on the exact solution,
>> > yet,
>> > > > and
>> > > > > > > some
>> > > > > > > > >> of
>> > > > > > > > >>> the changes might also have side effects that are hard
>> to
>> > > > > predict,
>> > > > > > > so I
>> > > > > > > > >> am
>> > > > > > > > >>> not sure we should rush this in.
>> > > > > > > > >>>
>> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
>> > Metaspace"
>> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
>> release?
>> > > It
>> > > > > > > changes
>> > > > > > > > >>> setups, takes away memory from heap / managed memory on
>> > > > existing
>> > > > > > > setups
>> > > > > > > > >>> that keep their flink-conf.yaml.
>> > > > > > > > >>>
>> > > > > > > > >>> We may need to unblock the release form these two issues
>> > and
>> > > > > think
>> > > > > > > > about
>> > > > > > > > >>> having 1.10.2 in the near future.
>> > > > > > > > >>>
>> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <carp84@gmail.com
>> >
>> > > > wrote:
>> > > > > > > > >>>
>> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these
>> > two.
>> > > > > > > > >>>>
>> > > > > > > > >>>> Best Regards,
>> > > > > > > > >>>> Yu
>> > > > > > > > >>>>
>> > > > > > > > >>>>
>> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <
>> imjark@gmail.com>
>> > > > wrote:
>> > > > > > > > >>>>
>> > > > > > > > >>>>> Thanks for driving this release, Yu!
>> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> From the Table SQL module, I think we should also try
>> to
>> > > get
>> > > > in
>> > > > > > the
>> > > > > > > > >>>>> following issues:
>> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
>> > > parameters
>> > > > > from
>> > > > > > > SQL
>> > > > > > > > >> CLI
>> > > > > > > > >>>>> environment
>> > > > > > > > >>>>> this allows users to set e.g. statebackend, watermark
>> > > > interval,
>> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
>> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
>> > > > exception
>> > > > > is
>> > > > > > > > >> thrown
>> > > > > > > > >>>>> this will improve much experience when using SQL CLI
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> Best,
>> > > > > > > > >>>>> Jark
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <carp84@gmail.com
>> >
>> > > > wrote:
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1
>> into
>> > > > > > > FLINK-16225
>> > > > > > > > >>>> fix
>> > > > > > > > >>>>>> versions and promoted its priority to Critical. Will
>> > also
>> > > > > watch
>> > > > > > > the
>> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> Best Regards,
>> > > > > > > > >>>>>> Yu
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
>> > > > > > > azagrebin@apache.org
>> > > > > > > > >
>> > > > > > > > >>>>>> wrote:
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>> Hi Yu,
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release
>> discussion!
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> Apart from
>> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
>> Metaspace
>> > to
>> > > > > > > minimise
>> > > > > > > > >>>> its
>> > > > > > > > >>>>>>> OutOfMemoryError
>> > > > > > > > >>>>>>> which should be merged soon
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> I think we should also try to get in the following
>> > > issues:
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be
>> > handled
>> > > > as
>> > > > > > > Fatal
>> > > > > > > > >>>>> Error
>> > > > > > > > >>>>>> in
>> > > > > > > > >>>>>>> TaskManager
>> > > > > > > > >>>>>>> This should solve the Metaspace problem even in a
>> > better
>> > > > way
>> > > > > > > > because
>> > > > > > > > >>>>> OOM
>> > > > > > > > >>>>>>> failure should point users to the docs immediately
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
>> lifetime
>> > > of
>> > > > a
>> > > > > > slot
>> > > > > > > > >>>>>>> This should give a better protection against class
>> > > loading
>> > > > > > leaks
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
>> submitting
>> > > > batch
>> > > > > > job
>> > > > > > > > >>>>>> (instead
>> > > > > > > > >>>>>>> of AskTimeoutException)
>> > > > > > > > >>>>>>> This problem has recently happened for multiple
>> users
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> Best,
>> > > > > > > > >>>>>>> Andrey
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
>> > > > > > > > jingsonglee0@gmail.com>
>> > > > > > > > >>>>>>> wrote:
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
>> > > > release.
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> Some issues are very important, Users are looking
>> > > forward
>> > > > to
>> > > > > > > them.
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> Best,
>> > > > > > > > >>>>>>>> Jingsong Lee
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
>> > > > > > karmagyz@gmail.com>
>> > > > > > > > >>>>> wrote:
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>> Best,
>> > > > > > > > >>>>>>>>> Yangze Guo
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
>> > > > > > > > >>>>> tonysong820@gmail.com>
>> > > > > > > > >>>>>>>>> wrote:
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>> Yu,
>> > > > > > > > >>>>>>>>>> Thanks for the explanation.
>> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some
>> > inputs
>> > > > for
>> > > > > > > > >>>>>>> prioritizing
>> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>> Thank you~
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>> Xintong Song
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
>> > > > carp84@gmail.com>
>> > > > > > > > >>>> wrote:
>> > > > > > > > >>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> According to the history, the first patch
>> release
>> > of
>> > > a
>> > > > > > major
>> > > > > > > > >>>>>>> version
>> > > > > > > > >>>>>>>>> will
>> > > > > > > > >>>>>>>>>>> take ~1month from discussion started, depending
>> on
>> > > the
>> > > > > > speed
>> > > > > > > > >>>> of
>> > > > > > > > >>>>>>>> blocker
>> > > > > > > > >>>>>>>>>>> issue resolving:
>> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
>> > released
>> > > > on
>> > > > > > Jul
>> > > > > > > > >>>>> 3rd
>> > > > > > > > >>>>>>> [2]
>> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
>> > released
>> > > > on
>> > > > > > Oct
>> > > > > > > > >>>>> 19th
>> > > > > > > > >>>>>>> [4]
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> We won't rush to match the history of course,
>> but
>> > > could
>> > > > > use
>> > > > > > > > >>>> it
>> > > > > > > > >>>>>> as a
>> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me know
>> if
>> > any
>> > > > > > > > >>>> concerns
>> > > > > > > > >>>>>>>> Xintong.
>> > > > > > > > >>>>>>>>>>> Thanks.
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> Best Regards,
>> > > > > > > > >>>>>>>>>>> Yu
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> [1]
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>> > > > > > > > >>>>>>>>>>> [2]
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>> > > > > > > > >>>>>>>>>>> [3]
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>> > > > > > > > >>>>>>>>>>> [4]
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
>> > > > > > > > >>>>>> tonysong820@gmail.com>
>> > > > > > > > >>>>>>>>> wrote:
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering
>> to be
>> > > the
>> > > > > > > > >>>>> release
>> > > > > > > > >>>>>>>>> manager.
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
>> > > 1.10.1?
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> Thank you~
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> Xintong Song
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
>> > > > > > > > >>>>>>>>>>>>> +1 on my side
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>> Best,
>> > > > > > > > >>>>>>>>>>>>> Zhijiang
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > >
>> > ------------------------------------------------------------------
>> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also
>> > keep
>> > > > an
>> > > > > > > > >>>> eye
>> > > > > > > > >>>>> on
>> > > > > > > > >>>>>>>> these
>> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>> Best Regards,
>> > > > > > > > >>>>>>>>>>>>> Yu
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
>> > > > > > > > >>>>> hequn@apache.org>
>> > > > > > > > >>>>>>>>> wrote:
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
>> > > > volunteer
>> > > > > > > > >>>> as
>> > > > > > > > >>>>>> the
>> > > > > > > > >>>>>>>>> release
>> > > > > > > > >>>>>>>>>>>>>> manager!
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which
>> are
>> > > > > marked
>> > > > > > > > >>>>> as
>> > > > > > > > >>>>>> a
>> > > > > > > > >>>>>>>>> blocker:
>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in
>> > NOTICE
>> > > > > files
>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
>> > > usrlib
>> > > > > > > > >>>>>> directory
>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>> > > > > > > > >>>>> ClassNotFoundException
>> > > > > > > > >>>>>>> when
>> > > > > > > > >>>>>>>>> use
>> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
>> > > > submitting
>> > > > > > > > >>>>> batch
>> > > > > > > > >>>>>>> job
>> > > > > > > > >>>>>>>>>>>> (instead
>> > > > > > > > >>>>>>>>>>>>> of
>> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> Best,
>> > > > > > > > >>>>>>>>>>>>>> Hequn
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> [1]
>> > > > > > > > >>>>>>>>>
>> > > > > > >
>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>> > > > > > > > >>>> carp84@gmail.com>
>> > > > > > > > >>>>>>>> wrote:
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful.
>> Thanks a
>> > > lot!
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
>> > > > > > > > >>>>>>>>>>>>>>> Yu
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
>> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>> wrote:
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I
>> would
>> > > > like
>> > > > > > > > >>>>> to
>> > > > > > > > >>>>>>> give
>> > > > > > > > >>>>>>>>> you a
>> > > > > > > > >>>>>>>>>>>>> hand
>> > > > > > > > >>>>>>>>>>>>>> at
>> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If
>> you
>> > > > > > > > >>>> need)
>> > > > > > > > >>>>>> :)
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>> Best,
>> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
>> > > 下午5:49写道：
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
>> > released
>> > > > > > > > >>>>> Flink
>> > > > > > > > >>>>>>>>> 1.10.0. We
>> > > > > > > > >>>>>>>>>>>>>> already
>> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
>> improvements/bugs
>> > in
>> > > > > > > > >>>> the
>> > > > > > > > >>>>>>>>>>> release-1.10
>> > > > > > > > >>>>>>>>>>>>>>> branch,
>> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release
>> > > cycle.
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license
>> and
>> > > > > > > > >>>> notice
>> > > > > > > > >>>>>>> file
>> > > > > > > > >>>>>>>> in
>> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
>> > > > > > > > >>>>>>>>>>>>>>>>> module
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
>> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be
>> known
>> > > in
>> > > > > > > > >>>>>>>>> HiveCatalog
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336
>> ArrayIndexOufOBoundsException in
>> > > > > > > > >>>>>>>> TypeExtractor
>> > > > > > > > >>>>>>>>>>> when
>> > > > > > > > >>>>>>>>>>>>>>> mapping
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed
>> if
>> > > > > > > > >>>> running
>> > > > > > > > >>>>>> in
>> > > > > > > > >>>>>>>>> blink
>> > > > > > > > >>>>>>>>>>>>> planner
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are
>> not
>> > > > > > > > >>>> reset
>> > > > > > > > >>>>>> on
>> > > > > > > > >>>>>>>> task
>> > > > > > > > >>>>>>>>>>>>> recovery
>> > > > > > > > >>>>>>>>>>>>>> in
>> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function
>> using
>> > > sql
>> > > > > > > > >>>>>>> casuing
>> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues
>> > > should
>> > > > > > > > >>>> be
>> > > > > > > > >>>>>>>> merged
>> > > > > > > > >>>>>>>>>>> before
>> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
>> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
>> issue):
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace
>> > OOM
>> > > > > > > > >>>>> error
>> > > > > > > > >>>>>> on
>> > > > > > > > >>>>>>>>>>> repeated
>> > > > > > > > >>>>>>>>>>>>> job
>> > > > > > > > >>>>>>>>>>>>>>>>> submission
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
>> JVM
>> > > > > > > > >>>>>> Metaspace
>> > > > > > > > >>>>>>> to
>> > > > > > > > >>>>>>>>>>>> minimise
>> > > > > > > > >>>>>>>>>>>>>> its
>> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
>> > > > > > > > >>>>> aggregate
>> > > > > > > > >>>>>>>>> results
>> > > > > > > > >>>>>>>>>>> with
>> > > > > > > > >>>>>>>>>>>>>> state
>> > > > > > > > >>>>>>>>>>>>>>>>> clean up
>> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not
>> extract
>> > > > > > > > >>>> correct
>> > > > > > > > >>>>>>>> unique
>> > > > > > > > >>>>>>>>> key
>> > > > > > > > >>>>>>>>>>>> for
>> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager
>> and
>> > > kick
>> > > > > > > > >>>>> off
>> > > > > > > > >>>>>>> the
>> > > > > > > > >>>>>>>>>>> release
>> > > > > > > > >>>>>>>>>>>>>>> process
>> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do
>> you
>> > > > > > > > >>>> think?
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
>> blocker
>> > > > > > > > >>>> issues
>> > > > > > > > >>>>>>> need
>> > > > > > > > >>>>>>>>> to be
>> > > > > > > > >>>>>>>>>>>>> fixed
>> > > > > > > > >>>>>>>>>>>>>> in
>> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
>> > > > > > > > >>>>>>>>>>>>>>>>> Yu
>> > > > > > > > >>>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>>
>> > > > > > > > >>>>>>>>>>>
>> > > > > > > > >>>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>> --
>> > > > > > > > >>>>>>>> Best, Jingsong Lee
>> > > > > > > > >>>>>>>>
>> > > > > > > > >>>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>
>> > > > > > > > >>>>
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the updates Till!
For FLINK-16018, maybe we could create two sub-tasks for easy and complete
fix separately, and only include the easy one in 1.10.1? Or please just
feel free to postpone the whole task to 1.10.2 if "all or nothing" policy
is preferred (smile). Thanks.
Best Regards,
Yu
On Tue, 17 Mar 2020 at 23:33, Till Rohrmann <tr...@apache.org> wrote:
> +1 for a soonish bug fix release. Thanks for volunteering as our release
> manager Yu.
>
> I think we can soon merge the increase of metaspace size and improving the
> error message. The assumption is that we currently don't have too many
> small Flink 1.10 deployments with a process size <= 1GB. Of course, the
> sooner we release the bug fix release, the fewer deployments will be
> affected by this change.
>
> For FLINK-16018, I think there would be an easy solution which we could
> include in the bug fix release. The proper fix will most likely take a bit
> longer, though.
>
> Cheers,
> Till
>
> On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <az...@apache.org>
> wrote:
>
> > > @Andrey and @Xintong - could we have a quick poll on the user mailing
> > list
> > > about increasing the metaspace size in Flink 1.10.1? Specifically
> asking
> > > for who has very small TM setups?
> >
> > There has been a survey about this topic since 10 days:
> >
> > `[Survey] Default size for the new JVM Metaspace limit in 1.10`
> > I can ask there about the small TM setups.
> >
> > On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
> >
> > > Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore
> > with
> > > memory state backends
> > >
> > > Let me recompile the watching list with recent feedbacks. There're
> > totally
> > > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
> > > already resolved and 31 left, and the below ones are watched (meaning
> > that
> > > once the below ones got fixed, we will kick of the releasing with left
> > ones
> > > postponed to 1.10.2 unless objections):
> > >
> > > * Blockers (7)
> > > - [Under Discussion] FLINK-16018 Improve error reporting when
> > submitting
> > > batch job (instead of AskTimeoutException)
> > > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM
> error
> > > on repeated job submission
> > > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> > ClassNotFoundException
> > > when use flink-sql-connector-elasticsearch7
> > > - [PR Submitted] FLINK-16262 Class loader problem with
> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > > minimise its OutOfMemoryError
> > > - [Open] FLINK-16454 Update the copyright year in NOTICE files
> > > - [Open] FLINK-16576 State inconsistency on restore with memory state
> > > backends
> > >
> > > * Critical (4)
> > > - [Open] FLINK-16047 Blink planner produces wrong aggregate results
> > with
> > > state clean up
> > > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> > unique
> > > key for UpsertStreamTableSink
> > > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> > > handled as Fatal Error in TaskManager
> > > - [Open] FLINK-16408 Bind user code class loader to lifetime of a
> slot
> > >
> > > Please raise your hand if you find any other issues should be put into
> > this
> > > list, thanks.
> > >
> > > Please also note that 1.10.2 version is already created (thanks for the
> > > help @jincheng), and feel free to adjust/assign fix version to it when
> > > necessary.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
> > >
> > > > @Andrey and @Xintong - could we have a quick poll on the user mailing
> > > list
> > > > about increasing the metaspace size in Flink 1.10.1? Specifically
> > asking
> > > > for who has very small TM setups?
> > > >
> > > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
> > > >
> > > > > Thanks for the reminder Stephan and the inputs/discussion Andrey
> and
> > > > > Xintong. I totally agree that we should try best not to introduce
> new
> > > > > issues when resolving the existing one. Will watch closely about
> the
> > > > > discussion and conclusion around metaspace configuration.
> > > > >
> > > > > From the point as a developer instead of RM, I'm optimistic that
> > > > reserving
> > > > > 160MB more memory to JVM metaspace by default won't cause a big
> > > problem,
> > > > > based on the assumption that nowadays assigning 2GB/4GB memory per
> TM
> > > is
> > > > > the common case.
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > > >
> > > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > I think in FLINK-16406 we have not only increased the default
> value
> > > for
> > > > > > metaspace, but also increased the 'process.size' in default
> > > > > > 'flink-conf.yaml'.
> > > > > >
> > > > > > That means:
> > > > > >
> > > > > > - For people using the default 'flink-conf.yaml', it's not
> > really
> > > > > taking
> > > > > > memory from heap / managed for metaspace.
> > > > > > - For people using custom configuration files:
> > > > > > - If they are specifying 'flink.size', then metaspace does
> > not
> > > > > really
> > > > > > affect the flink memory internal components (heap, managed,
> > > > etc.).
> > > > > > - If they are specifying 'process.size' (most likely active
> > > > > setups),
> > > > > > then they will have less heap / managed memory, or worse,
> > their
> > > > > > configuration may not work without adjustment.
> > > > > > - For a TM with large process memory (e.g., 4 GB or
> more),
> > > it
> > > > is
> > > > > > probably ok that the metaspace takes 160 MB more memory.
> > > > > > - For a TM with smaller total memory (1 or 2 GB) the
> > > metaspace
> > > > > > increase is more likely to cause problem.
> > > > > >
> > > > > > So basically only people have small 'process.size' in custom
> config
> > > > file
> > > > > > are really affected. I'm not sure what is the proportion of such
> > use
> > > > > cases
> > > > > > though. (From questions asked on the user ML, probably not much).
> > > > > >
> > > > > > Thank you~
> > > > > >
> > > > > > Xintong Song
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org>
> > > > wrote:
> > > > > >
> > > > > > > No need to revert it now - I am not saying it should not go
> into
> > > > > 1.10.1,
> > > > > > I
> > > > > > > am just saying this is not clear to me yet.
> > > > > > >
> > > > > > > We have to trade off the fact that we may break some
> deployments
> > > with
> > > > > the
> > > > > > > fact that we will "safe" a lot of new deployments.
> > > > > > > I simply lack the data points / insight at the moment to
> > understand
> > > > how
> > > > > > > significant both cases are, meaning how many users would be
> > > affected
> > > > > and
> > > > > > > how badly.
> > > > > > >
> > > > > > > Independent of that, improving the error message is always
> > helpful.
> > > > > > >
> > > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > > > > > azagrebin.apache@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > > - For 1.10.1 I am not completely sure, because users
> expect
> > > to
> > > > > > > upgrade
> > > > > > > > > that without config adjustments. That might not be possible
> > > with
> > > > > that
> > > > > > > > > change.
> > > > > > > >
> > > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
> > > improve
> > > > > > error
> > > > > > > > message and docs.
> > > > > > > >
> > > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org>
> > > wrote:
> > > > > > > > >
> > > > > > > > > @Andrey about the increase in metaspace size
> > > > > > > > > - I have no concerns for 1.11.0.
> > > > > > > > > - For 1.10.1 I am not completely sure, because users
> expect
> > > to
> > > > > > > upgrade
> > > > > > > > > that without config adjustments. That might not be possible
> > > with
> > > > > that
> > > > > > > > > change.
> > > > > > > > >
> > > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > > > > > > azagrebin.apache@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >>
> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error
> > on
> > > > > > repeated
> > > > > > > > >> job”
> > > > > > > > >>
> > > > > > > > >> My understanding that the issue is basically covered by:
> > > > > > > > >>
> > > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled
> as
> > > > Fatal
> > > > > > > Error
> > > > > > > > >> in TaskManager
> > > > > > > > >> no full consensus there but improving error message for
> > > > existing
> > > > > > > task
> > > > > > > > >> thread fatal handling could be done at least
> > > > > > > > >>
> > > > > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace
> to
> > > > > minimise
> > > > > > > its
> > > > > > > > >> OutOfMemoryError
> > > > > > > > >> see further
> > > > > > > > >>
> > > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
> > > > dynamically
> > > > > > > > loaded
> > > > > > > > >> AWS connectors
> > > > > > > > >> not sure whether this is a blocker but looks close to be
> > > > resolved
> > > > > > > > >>
> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> > Metaspace"
> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
> release?
> > > It
> > > > > > > changes
> > > > > > > > >>> setups, takes away memory from heap / managed memory on
> > > > existing
> > > > > > > setups
> > > > > > > > >>> that keep their flink-conf.yaml.
> > > > > > > > >>
> > > > > > > > >> My understanding was that increasing to 256m resolved the
> > > > reported
> > > > > > > > problems
> > > > > > > > >> and we decided to make the change so I have merged it
> today
> > as
> > > > > there
> > > > > > > > were
> > > > > > > > >> no more concerns.
> > > > > > > > >> If there are concerns I can revert it.
> > > > > > > > >>
> > > > > > > > >> On the other hand, I think improving the message error
> with
> > > > > > reference
> > > > > > > to
> > > > > > > > >> the metaspace option should help the most
> > > > > > > > >> because user would not have to read all docs to fix it
> > > > > > > > >> then maybe this change is not even needed.
> > > > > > > > >>
> > > > > > > > >> Best,
> > > > > > > > >> Andrey
> > > > > > > > >>
> > > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <sewen@apache.org
> >
> > > > wrote:
> > > > > > > > >>>
> > > > > > > > >>> Good idea to go ahead with 1.10.1
> > > > > > > > >>>
> > > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error
> > on
> > > > > > repeated
> > > > > > > > >> job"
> > > > > > > > >>> - I don't think we have consensus on the exact solution,
> > yet,
> > > > and
> > > > > > > some
> > > > > > > > >> of
> > > > > > > > >>> the changes might also have side effects that are hard to
> > > > > predict,
> > > > > > > so I
> > > > > > > > >> am
> > > > > > > > >>> not sure we should rush this in.
> > > > > > > > >>>
> > > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> > Metaspace"
> > > > > > > > >>> - Have we consensus that this is okay for a bugfix
> release?
> > > It
> > > > > > > changes
> > > > > > > > >>> setups, takes away memory from heap / managed memory on
> > > > existing
> > > > > > > setups
> > > > > > > > >>> that keep their flink-conf.yaml.
> > > > > > > > >>>
> > > > > > > > >>> We may need to unblock the release form these two issues
> > and
> > > > > think
> > > > > > > > about
> > > > > > > > >>> having 1.10.2 in the near future.
> > > > > > > > >>>
> > > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com>
> > > > wrote:
> > > > > > > > >>>
> > > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these
> > two.
> > > > > > > > >>>>
> > > > > > > > >>>> Best Regards,
> > > > > > > > >>>> Yu
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <imjark@gmail.com
> >
> > > > wrote:
> > > > > > > > >>>>
> > > > > > > > >>>>> Thanks for driving this release, Yu!
> > > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> > > > > > > > >>>>>
> > > > > > > > >>>>> From the Table SQL module, I think we should also try
> to
> > > get
> > > > in
> > > > > > the
> > > > > > > > >>>>> following issues:
> > > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
> > > parameters
> > > > > from
> > > > > > > SQL
> > > > > > > > >> CLI
> > > > > > > > >>>>> environment
> > > > > > > > >>>>> this allows users to set e.g. statebackend, watermark
> > > > interval,
> > > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
> > > > exception
> > > > > is
> > > > > > > > >> thrown
> > > > > > > > >>>>> this will improve much experience when using SQL CLI
> > > > > > > > >>>>>
> > > > > > > > >>>>> Best,
> > > > > > > > >>>>> Jark
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com>
> > > > wrote:
> > > > > > > > >>>>>
> > > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1
> into
> > > > > > > FLINK-16225
> > > > > > > > >>>> fix
> > > > > > > > >>>>>> versions and promoted its priority to Critical. Will
> > also
> > > > > watch
> > > > > > > the
> > > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Best Regards,
> > > > > > > > >>>>>> Yu
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > > > > > > azagrebin@apache.org
> > > > > > > > >
> > > > > > > > >>>>>> wrote:
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>> Hi Yu,
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Apart from
> > > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM
> Metaspace
> > to
> > > > > > > minimise
> > > > > > > > >>>> its
> > > > > > > > >>>>>>> OutOfMemoryError
> > > > > > > > >>>>>>> which should be merged soon
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> I think we should also try to get in the following
> > > issues:
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be
> > handled
> > > > as
> > > > > > > Fatal
> > > > > > > > >>>>> Error
> > > > > > > > >>>>>> in
> > > > > > > > >>>>>>> TaskManager
> > > > > > > > >>>>>>> This should solve the Metaspace problem even in a
> > better
> > > > way
> > > > > > > > because
> > > > > > > > >>>>> OOM
> > > > > > > > >>>>>>> failure should point users to the docs immediately
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to
> lifetime
> > > of
> > > > a
> > > > > > slot
> > > > > > > > >>>>>>> This should give a better protection against class
> > > loading
> > > > > > leaks
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when
> submitting
> > > > batch
> > > > > > job
> > > > > > > > >>>>>> (instead
> > > > > > > > >>>>>>> of AskTimeoutException)
> > > > > > > > >>>>>>> This problem has recently happened for multiple users
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> Best,
> > > > > > > > >>>>>>> Andrey
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > > > > > > jingsonglee0@gmail.com>
> > > > > > > > >>>>>>> wrote:
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
> > > > release.
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> Some issues are very important, Users are looking
> > > forward
> > > > to
> > > > > > > them.
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> Best,
> > > > > > > > >>>>>>>> Jingsong Lee
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> > > > > > karmagyz@gmail.com>
> > > > > > > > >>>>> wrote:
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>> Best,
> > > > > > > > >>>>>>>>> Yangze Guo
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > > > > > > >>>>> tonysong820@gmail.com>
> > > > > > > > >>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Yu,
> > > > > > > > >>>>>>>>>> Thanks for the explanation.
> > > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some
> > inputs
> > > > for
> > > > > > > > >>>>>>> prioritizing
> > > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Thank you~
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> Xintong Song
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> > > > carp84@gmail.com>
> > > > > > > > >>>> wrote:
> > > > > > > > >>>>>>>>>>
> > > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> According to the history, the first patch release
> > of
> > > a
> > > > > > major
> > > > > > > > >>>>>>> version
> > > > > > > > >>>>>>>>> will
> > > > > > > > >>>>>>>>>>> take ~1month from discussion started, depending
> on
> > > the
> > > > > > speed
> > > > > > > > >>>> of
> > > > > > > > >>>>>>>> blocker
> > > > > > > > >>>>>>>>>>> issue resolving:
> > > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
> > released
> > > > on
> > > > > > Jul
> > > > > > > > >>>>> 3rd
> > > > > > > > >>>>>>> [2]
> > > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
> > released
> > > > on
> > > > > > Oct
> > > > > > > > >>>>> 19th
> > > > > > > > >>>>>>> [4]
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> We won't rush to match the history of course, but
> > > could
> > > > > use
> > > > > > > > >>>> it
> > > > > > > > >>>>>> as a
> > > > > > > > >>>>>>>>>>> reference. And please feel free to let me know if
> > any
> > > > > > > > >>>> concerns
> > > > > > > > >>>>>>>> Xintong.
> > > > > > > > >>>>>>>>>>> Thanks.
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> Best Regards,
> > > > > > > > >>>>>>>>>>> Yu
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> [1]
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > > > >>>>>>>>>>> [2]
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > > > >>>>>>>>>>> [3]
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > > > >>>>>>>>>>> [4]
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > > > > > > >>>>>> tonysong820@gmail.com>
> > > > > > > > >>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to
> be
> > > the
> > > > > > > > >>>>> release
> > > > > > > > >>>>>>>>> manager.
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> +1 for the proposal.
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
> > > 1.10.1?
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> Thank you~
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> Xintong Song
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > > > > > > >>>>>>>>>>>>> +1 on my side
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>> Best,
> > > > > > > > >>>>>>>>>>>>> Zhijiang
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > >
> > ------------------------------------------------------------------
> > > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also
> > keep
> > > > an
> > > > > > > > >>>> eye
> > > > > > > > >>>>> on
> > > > > > > > >>>>>>>> these
> > > > > > > > >>>>>>>>>>>>> existing blocker issues.
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>> Best Regards,
> > > > > > > > >>>>>>>>>>>>> Yu
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > > > > > > >>>>> hequn@apache.org>
> > > > > > > > >>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> Hi Yu,
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
> > > > volunteer
> > > > > > > > >>>> as
> > > > > > > > >>>>>> the
> > > > > > > > >>>>>>>>> release
> > > > > > > > >>>>>>>>>>>>>> manager!
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which
> are
> > > > > marked
> > > > > > > > >>>>> as
> > > > > > > > >>>>>> a
> > > > > > > > >>>>>>>>> blocker:
> > > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in
> > NOTICE
> > > > > files
> > > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
> > > usrlib
> > > > > > > > >>>>>> directory
> > > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > > > > > > >>>>> ClassNotFoundException
> > > > > > > > >>>>>>> when
> > > > > > > > >>>>>>>>> use
> > > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
> > > > submitting
> > > > > > > > >>>>> batch
> > > > > > > > >>>>>>> job
> > > > > > > > >>>>>>>>>>>> (instead
> > > > > > > > >>>>>>>>>>>>> of
> > > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> Best,
> > > > > > > > >>>>>>>>>>>>>> Hequn
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> [1]
> > > > > > > > >>>>>>>>>
> > > > > > >
> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > > > > > > >>>> carp84@gmail.com>
> > > > > > > > >>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks
> a
> > > lot!
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>> Best Regards,
> > > > > > > > >>>>>>>>>>>>>>> Yu
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>> wrote:
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I
> would
> > > > like
> > > > > > > > >>>>> to
> > > > > > > > >>>>>>> give
> > > > > > > > >>>>>>>>> you a
> > > > > > > > >>>>>>>>>>>>> hand
> > > > > > > > >>>>>>>>>>>>>> at
> > > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If
> you
> > > > > > > > >>>> need)
> > > > > > > > >>>>>> :)
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>> Best,
> > > > > > > > >>>>>>>>>>>>>>>> Jincheng
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
> > > 下午5:49写道：
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
> > released
> > > > > > > > >>>>> Flink
> > > > > > > > >>>>>>>>> 1.10.0. We
> > > > > > > > >>>>>>>>>>>>>> already
> > > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved
> improvements/bugs
> > in
> > > > > > > > >>>> the
> > > > > > > > >>>>>>>>>>> release-1.10
> > > > > > > > >>>>>>>>>>>>>>> branch,
> > > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release
> > > cycle.
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license
> and
> > > > > > > > >>>> notice
> > > > > > > > >>>>>>> file
> > > > > > > > >>>>>>>> in
> > > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > > > > > > >>>>>>>>>>>>>>>>> module
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be
> known
> > > in
> > > > > > > > >>>>>>>>> HiveCatalog
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException
> in
> > > > > > > > >>>>>>>> TypeExtractor
> > > > > > > > >>>>>>>>>>> when
> > > > > > > > >>>>>>>>>>>>>>> mapping
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > > > > > > >>>> running
> > > > > > > > >>>>>> in
> > > > > > > > >>>>>>>>> blink
> > > > > > > > >>>>>>>>>>>>> planner
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are
> not
> > > > > > > > >>>> reset
> > > > > > > > >>>>>> on
> > > > > > > > >>>>>>>> task
> > > > > > > > >>>>>>>>>>>>> recovery
> > > > > > > > >>>>>>>>>>>>>> in
> > > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function
> using
> > > sql
> > > > > > > > >>>>>>> casuing
> > > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues
> > > should
> > > > > > > > >>>> be
> > > > > > > > >>>>>>>> merged
> > > > > > > > >>>>>>>>>>> before
> > > > > > > > >>>>>>>>>>>>>>> 1.10.1
> > > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM
> issue):
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace
> > OOM
> > > > > > > > >>>>> error
> > > > > > > > >>>>>> on
> > > > > > > > >>>>>>>>>>> repeated
> > > > > > > > >>>>>>>>>>>>> job
> > > > > > > > >>>>>>>>>>>>>>>>> submission
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for
> JVM
> > > > > > > > >>>>>> Metaspace
> > > > > > > > >>>>>>> to
> > > > > > > > >>>>>>>>>>>> minimise
> > > > > > > > >>>>>>>>>>>>>> its
> > > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > > > > > > >>>>> aggregate
> > > > > > > > >>>>>>>>> results
> > > > > > > > >>>>>>>>>>> with
> > > > > > > > >>>>>>>>>>>>>> state
> > > > > > > > >>>>>>>>>>>>>>>>> clean up
> > > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > > > > > > >>>> correct
> > > > > > > > >>>>>>>> unique
> > > > > > > > >>>>>>>>> key
> > > > > > > > >>>>>>>>>>>> for
> > > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager
> and
> > > kick
> > > > > > > > >>>>> off
> > > > > > > > >>>>>>> the
> > > > > > > > >>>>>>>>>>> release
> > > > > > > > >>>>>>>>>>>>>>> process
> > > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > > > > > > >>>> think?
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing
> blocker
> > > > > > > > >>>> issues
> > > > > > > > >>>>>>> need
> > > > > > > > >>>>>>>>> to be
> > > > > > > > >>>>>>>>>>>>> fixed
> > > > > > > > >>>>>>>>>>>>>> in
> > > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > > > > > > >>>>>>>>>>>>>>>>> Yu
> > > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>>
> > > > > > > > >>>>>>>>>>>
> > > > > > > > >>>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>> --
> > > > > > > > >>>>>>>> Best, Jingsong Lee
> > > > > > > > >>>>>>>>
> > > > > > > > >>>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Till Rohrmann <tr...@apache.org>.
+1 for a soonish bug fix release. Thanks for volunteering as our release
manager Yu.
I think we can soon merge the increase of metaspace size and improving the
error message. The assumption is that we currently don't have too many
small Flink 1.10 deployments with a process size <= 1GB. Of course, the
sooner we release the bug fix release, the fewer deployments will be
affected by this change.
For FLINK-16018, I think there would be an easy solution which we could
include in the bug fix release. The proper fix will most likely take a bit
longer, though.
Cheers,
Till
On Fri, Mar 13, 2020 at 8:08 PM Andrey Zagrebin <az...@apache.org>
wrote:
> > @Andrey and @Xintong - could we have a quick poll on the user mailing
> list
> > about increasing the metaspace size in Flink 1.10.1? Specifically asking
> > for who has very small TM setups?
>
> There has been a survey about this topic since 10 days:
>
> `[Survey] Default size for the new JVM Metaspace limit in 1.10`
> I can ask there about the small TM setups.
>
> On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
>
> > Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore
> with
> > memory state backends
> >
> > Let me recompile the watching list with recent feedbacks. There're
> totally
> > 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
> > already resolved and 31 left, and the below ones are watched (meaning
> that
> > once the below ones got fixed, we will kick of the releasing with left
> ones
> > postponed to 1.10.2 unless objections):
> >
> > * Blockers (7)
> > - [Under Discussion] FLINK-16018 Improve error reporting when
> submitting
> > batch job (instead of AskTimeoutException)
> > - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
> > on repeated job submission
> > - [PR Submitted] FLINK-16170 SearchTemplateRequest
> ClassNotFoundException
> > when use flink-sql-connector-elasticsearch7
> > - [PR Submitted] FLINK-16262 Class loader problem with
> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> > minimise its OutOfMemoryError
> > - [Open] FLINK-16454 Update the copyright year in NOTICE files
> > - [Open] FLINK-16576 State inconsistency on restore with memory state
> > backends
> >
> > * Critical (4)
> > - [Open] FLINK-16047 Blink planner produces wrong aggregate results
> with
> > state clean up
> > - [PR Submitted] FLINK-16070 Blink planner can not extract correct
> unique
> > key for UpsertStreamTableSink
> > - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> > handled as Fatal Error in TaskManager
> > - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
> >
> > Please raise your hand if you find any other issues should be put into
> this
> > list, thanks.
> >
> > Please also note that 1.10.2 version is already created (thanks for the
> > help @jincheng), and feel free to adjust/assign fix version to it when
> > necessary.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
> >
> > > @Andrey and @Xintong - could we have a quick poll on the user mailing
> > list
> > > about increasing the metaspace size in Flink 1.10.1? Specifically
> asking
> > > for who has very small TM setups?
> > >
> > > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
> > >
> > > > Thanks for the reminder Stephan and the inputs/discussion Andrey and
> > > > Xintong. I totally agree that we should try best not to introduce new
> > > > issues when resolving the existing one. Will watch closely about the
> > > > discussion and conclusion around metaspace configuration.
> > > >
> > > > From the point as a developer instead of RM, I'm optimistic that
> > > reserving
> > > > 160MB more memory to JVM metaspace by default won't cause a big
> > problem,
> > > > based on the assumption that nowadays assigning 2GB/4GB memory per TM
> > is
> > > > the common case.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com>
> > > wrote:
> > > >
> > > > > I think in FLINK-16406 we have not only increased the default value
> > for
> > > > > metaspace, but also increased the 'process.size' in default
> > > > > 'flink-conf.yaml'.
> > > > >
> > > > > That means:
> > > > >
> > > > > - For people using the default 'flink-conf.yaml', it's not
> really
> > > > taking
> > > > > memory from heap / managed for metaspace.
> > > > > - For people using custom configuration files:
> > > > > - If they are specifying 'flink.size', then metaspace does
> not
> > > > really
> > > > > affect the flink memory internal components (heap, managed,
> > > etc.).
> > > > > - If they are specifying 'process.size' (most likely active
> > > > setups),
> > > > > then they will have less heap / managed memory, or worse,
> their
> > > > > configuration may not work without adjustment.
> > > > > - For a TM with large process memory (e.g., 4 GB or more),
> > it
> > > is
> > > > > probably ok that the metaspace takes 160 MB more memory.
> > > > > - For a TM with smaller total memory (1 or 2 GB) the
> > metaspace
> > > > > increase is more likely to cause problem.
> > > > >
> > > > > So basically only people have small 'process.size' in custom config
> > > file
> > > > > are really affected. I'm not sure what is the proportion of such
> use
> > > > cases
> > > > > though. (From questions asked on the user ML, probably not much).
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org>
> > > wrote:
> > > > >
> > > > > > No need to revert it now - I am not saying it should not go into
> > > > 1.10.1,
> > > > > I
> > > > > > am just saying this is not clear to me yet.
> > > > > >
> > > > > > We have to trade off the fact that we may break some deployments
> > with
> > > > the
> > > > > > fact that we will "safe" a lot of new deployments.
> > > > > > I simply lack the data points / insight at the moment to
> understand
> > > how
> > > > > > significant both cases are, meaning how many users would be
> > affected
> > > > and
> > > > > > how badly.
> > > > > >
> > > > > > Independent of that, improving the error message is always
> helpful.
> > > > > >
> > > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > > > > azagrebin.apache@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > > - For 1.10.1 I am not completely sure, because users expect
> > to
> > > > > > upgrade
> > > > > > > > that without config adjustments. That might not be possible
> > with
> > > > that
> > > > > > > > change.
> > > > > > >
> > > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
> > improve
> > > > > error
> > > > > > > message and docs.
> > > > > > >
> > > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org>
> > wrote:
> > > > > > > >
> > > > > > > > @Andrey about the increase in metaspace size
> > > > > > > > - I have no concerns for 1.11.0.
> > > > > > > > - For 1.10.1 I am not completely sure, because users expect
> > to
> > > > > > upgrade
> > > > > > > > that without config adjustments. That might not be possible
> > with
> > > > that
> > > > > > > > change.
> > > > > > > >
> > > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > > > > > azagrebin.apache@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >>
> > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error
> on
> > > > > repeated
> > > > > > > >> job”
> > > > > > > >>
> > > > > > > >> My understanding that the issue is basically covered by:
> > > > > > > >>
> > > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> > > Fatal
> > > > > > Error
> > > > > > > >> in TaskManager
> > > > > > > >> no full consensus there but improving error message for
> > > existing
> > > > > > task
> > > > > > > >> thread fatal handling could be done at least
> > > > > > > >>
> > > > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace to
> > > > minimise
> > > > > > its
> > > > > > > >> OutOfMemoryError
> > > > > > > >> see further
> > > > > > > >>
> > > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
> > > dynamically
> > > > > > > loaded
> > > > > > > >> AWS connectors
> > > > > > > >> not sure whether this is a blocker but looks close to be
> > > resolved
> > > > > > > >>
> > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> Metaspace"
> > > > > > > >>> - Have we consensus that this is okay for a bugfix release?
> > It
> > > > > > changes
> > > > > > > >>> setups, takes away memory from heap / managed memory on
> > > existing
> > > > > > setups
> > > > > > > >>> that keep their flink-conf.yaml.
> > > > > > > >>
> > > > > > > >> My understanding was that increasing to 256m resolved the
> > > reported
> > > > > > > problems
> > > > > > > >> and we decided to make the change so I have merged it today
> as
> > > > there
> > > > > > > were
> > > > > > > >> no more concerns.
> > > > > > > >> If there are concerns I can revert it.
> > > > > > > >>
> > > > > > > >> On the other hand, I think improving the message error with
> > > > > reference
> > > > > > to
> > > > > > > >> the metaspace option should help the most
> > > > > > > >> because user would not have to read all docs to fix it
> > > > > > > >> then maybe this change is not even needed.
> > > > > > > >>
> > > > > > > >> Best,
> > > > > > > >> Andrey
> > > > > > > >>
> > > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org>
> > > wrote:
> > > > > > > >>>
> > > > > > > >>> Good idea to go ahead with 1.10.1
> > > > > > > >>>
> > > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error
> on
> > > > > repeated
> > > > > > > >> job"
> > > > > > > >>> - I don't think we have consensus on the exact solution,
> yet,
> > > and
> > > > > > some
> > > > > > > >> of
> > > > > > > >>> the changes might also have side effects that are hard to
> > > > predict,
> > > > > > so I
> > > > > > > >> am
> > > > > > > >>> not sure we should rush this in.
> > > > > > > >>>
> > > > > > > >>> About "FLINK-16406 Increase default value for JVM
> Metaspace"
> > > > > > > >>> - Have we consensus that this is okay for a bugfix release?
> > It
> > > > > > changes
> > > > > > > >>> setups, takes away memory from heap / managed memory on
> > > existing
> > > > > > setups
> > > > > > > >>> that keep their flink-conf.yaml.
> > > > > > > >>>
> > > > > > > >>> We may need to unblock the release form these two issues
> and
> > > > think
> > > > > > > about
> > > > > > > >>> having 1.10.2 in the near future.
> > > > > > > >>>
> > > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com>
> > > wrote:
> > > > > > > >>>
> > > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these
> two.
> > > > > > > >>>>
> > > > > > > >>>> Best Regards,
> > > > > > > >>>> Yu
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com>
> > > wrote:
> > > > > > > >>>>
> > > > > > > >>>>> Thanks for driving this release, Yu!
> > > > > > > >>>>> +1 to start 1.10.1 release cycle.
> > > > > > > >>>>>
> > > > > > > >>>>> From the Table SQL module, I think we should also try to
> > get
> > > in
> > > > > the
> > > > > > > >>>>> following issues:
> > > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
> > parameters
> > > > from
> > > > > > SQL
> > > > > > > >> CLI
> > > > > > > >>>>> environment
> > > > > > > >>>>> this allows users to set e.g. statebackend, watermark
> > > interval,
> > > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
> > > exception
> > > > is
> > > > > > > >> thrown
> > > > > > > >>>>> this will improve much experience when using SQL CLI
> > > > > > > >>>>>
> > > > > > > >>>>> Best,
> > > > > > > >>>>> Jark
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com>
> > > wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> > > > > > FLINK-16225
> > > > > > > >>>> fix
> > > > > > > >>>>>> versions and promoted its priority to Critical. Will
> also
> > > > watch
> > > > > > the
> > > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > > > > > >>>>>>
> > > > > > > >>>>>> Best Regards,
> > > > > > > >>>>>> Yu
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > > > > > azagrebin@apache.org
> > > > > > > >
> > > > > > > >>>>>> wrote:
> > > > > > > >>>>>>
> > > > > > > >>>>>>> Hi Yu,
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Apart from
> > > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace
> to
> > > > > > minimise
> > > > > > > >>>> its
> > > > > > > >>>>>>> OutOfMemoryError
> > > > > > > >>>>>>> which should be merged soon
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> I think we should also try to get in the following
> > issues:
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be
> handled
> > > as
> > > > > > Fatal
> > > > > > > >>>>> Error
> > > > > > > >>>>>> in
> > > > > > > >>>>>>> TaskManager
> > > > > > > >>>>>>> This should solve the Metaspace problem even in a
> better
> > > way
> > > > > > > because
> > > > > > > >>>>> OOM
> > > > > > > >>>>>>> failure should point users to the docs immediately
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime
> > of
> > > a
> > > > > slot
> > > > > > > >>>>>>> This should give a better protection against class
> > loading
> > > > > leaks
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting
> > > batch
> > > > > job
> > > > > > > >>>>>> (instead
> > > > > > > >>>>>>> of AskTimeoutException)
> > > > > > > >>>>>>> This problem has recently happened for multiple users
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> Best,
> > > > > > > >>>>>>> Andrey
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > > > > > jingsonglee0@gmail.com>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>
> > > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
> > > release.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Some issues are very important, Users are looking
> > forward
> > > to
> > > > > > them.
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> Best,
> > > > > > > >>>>>>>> Jingsong Lee
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> > > > > karmagyz@gmail.com>
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> Best,
> > > > > > > >>>>>>>>> Yangze Guo
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > > > > > >>>>> tonysong820@gmail.com>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Yu,
> > > > > > > >>>>>>>>>> Thanks for the explanation.
> > > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some
> inputs
> > > for
> > > > > > > >>>>>>> prioritizing
> > > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Thank you~
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> Xintong Song
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> > > carp84@gmail.com>
> > > > > > > >>>> wrote:
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> According to the history, the first patch release
> of
> > a
> > > > > major
> > > > > > > >>>>>>> version
> > > > > > > >>>>>>>>> will
> > > > > > > >>>>>>>>>>> take ~1month from discussion started, depending on
> > the
> > > > > speed
> > > > > > > >>>> of
> > > > > > > >>>>>>>> blocker
> > > > > > > >>>>>>>>>>> issue resolving:
> > > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1],
> released
> > > on
> > > > > Jul
> > > > > > > >>>>> 3rd
> > > > > > > >>>>>>> [2]
> > > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3],
> released
> > > on
> > > > > Oct
> > > > > > > >>>>> 19th
> > > > > > > >>>>>>> [4]
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> We won't rush to match the history of course, but
> > could
> > > > use
> > > > > > > >>>> it
> > > > > > > >>>>>> as a
> > > > > > > >>>>>>>>>>> reference. And please feel free to let me know if
> any
> > > > > > > >>>> concerns
> > > > > > > >>>>>>>> Xintong.
> > > > > > > >>>>>>>>>>> Thanks.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> Best Regards,
> > > > > > > >>>>>>>>>>> Yu
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> [1]
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > > >>>>>>>>>>> [2]
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > > >>>>>>>>>>> [3]
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > > >>>>>>>>>>> [4]
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > > > > > >>>>>> tonysong820@gmail.com>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be
> > the
> > > > > > > >>>>> release
> > > > > > > >>>>>>>>> manager.
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> +1 for the proposal.
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
> > 1.10.1?
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> Thank you~
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> Xintong Song
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > > > > > >>>>>>>>>>>>> +1 on my side
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> Best,
> > > > > > > >>>>>>>>>>>>> Zhijiang
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > >
> ------------------------------------------------------------------
> > > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also
> keep
> > > an
> > > > > > > >>>> eye
> > > > > > > >>>>> on
> > > > > > > >>>>>>>> these
> > > > > > > >>>>>>>>>>>>> existing blocker issues.
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> Best Regards,
> > > > > > > >>>>>>>>>>>>> Yu
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > > > > > >>>>> hequn@apache.org>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> Hi Yu,
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
> > > volunteer
> > > > > > > >>>> as
> > > > > > > >>>>>> the
> > > > > > > >>>>>>>>> release
> > > > > > > >>>>>>>>>>>>>> manager!
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which are
> > > > marked
> > > > > > > >>>>> as
> > > > > > > >>>>>> a
> > > > > > > >>>>>>>>> blocker:
> > > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in
> NOTICE
> > > > files
> > > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
> > usrlib
> > > > > > > >>>>>> directory
> > > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > > > > > >>>>> ClassNotFoundException
> > > > > > > >>>>>>> when
> > > > > > > >>>>>>>>> use
> > > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
> > > submitting
> > > > > > > >>>>> batch
> > > > > > > >>>>>>> job
> > > > > > > >>>>>>>>>>>> (instead
> > > > > > > >>>>>>>>>>>>> of
> > > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> Best,
> > > > > > > >>>>>>>>>>>>>> Hequn
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> [1]
> > > > > > > >>>>>>>>>
> > > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > > > > > >>>> carp84@gmail.com>
> > > > > > > >>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a
> > lot!
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>> Best Regards,
> > > > > > > >>>>>>>>>>>>>>> Yu
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would
> > > like
> > > > > > > >>>>> to
> > > > > > > >>>>>>> give
> > > > > > > >>>>>>>>> you a
> > > > > > > >>>>>>>>>>>>> hand
> > > > > > > >>>>>>>>>>>>>> at
> > > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > > > > > > >>>> need)
> > > > > > > >>>>>> :)
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>> Best,
> > > > > > > >>>>>>>>>>>>>>>> Jincheng
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
> > 下午5:49写道：
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> Hi All,
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we
> released
> > > > > > > >>>>> Flink
> > > > > > > >>>>>>>>> 1.10.0. We
> > > > > > > >>>>>>>>>>>>>> already
> > > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs
> in
> > > > > > > >>>> the
> > > > > > > >>>>>>>>>>> release-1.10
> > > > > > > >>>>>>>>>>>>>>> branch,
> > > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release
> > cycle.
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > > > > > > >>>> notice
> > > > > > > >>>>>>> file
> > > > > > > >>>>>>>> in
> > > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > > > > > >>>>>>>>>>>>>>>>> module
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > >>>>>>>>>>>>> flink-state-processor-api
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known
> > in
> > > > > > > >>>>>>>>> HiveCatalog
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > > > > >>>>>>>> TypeExtractor
> > > > > > > >>>>>>>>>>> when
> > > > > > > >>>>>>>>>>>>>>> mapping
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > > > > > >>>> running
> > > > > > > >>>>>> in
> > > > > > > >>>>>>>>> blink
> > > > > > > >>>>>>>>>>>>> planner
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > > > > > > >>>> reset
> > > > > > > >>>>>> on
> > > > > > > >>>>>>>> task
> > > > > > > >>>>>>>>>>>>> recovery
> > > > > > > >>>>>>>>>>>>>> in
> > > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using
> > sql
> > > > > > > >>>>>>> casuing
> > > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues
> > should
> > > > > > > >>>> be
> > > > > > > >>>>>>>> merged
> > > > > > > >>>>>>>>>>> before
> > > > > > > >>>>>>>>>>>>>>> 1.10.1
> > > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace
> OOM
> > > > > > > >>>>> error
> > > > > > > >>>>>> on
> > > > > > > >>>>>>>>>>> repeated
> > > > > > > >>>>>>>>>>>>> job
> > > > > > > >>>>>>>>>>>>>>>>> submission
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > > > > > > >>>>>> Metaspace
> > > > > > > >>>>>>> to
> > > > > > > >>>>>>>>>>>> minimise
> > > > > > > >>>>>>>>>>>>>> its
> > > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > > > > > >>>>> aggregate
> > > > > > > >>>>>>>>> results
> > > > > > > >>>>>>>>>>> with
> > > > > > > >>>>>>>>>>>>>> state
> > > > > > > >>>>>>>>>>>>>>>>> clean up
> > > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > > > > > >>>> correct
> > > > > > > >>>>>>>> unique
> > > > > > > >>>>>>>>> key
> > > > > > > >>>>>>>>>>>> for
> > > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and
> > kick
> > > > > > > >>>>> off
> > > > > > > >>>>>>> the
> > > > > > > >>>>>>>>>>> release
> > > > > > > >>>>>>>>>>>>>>> process
> > > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > > > > > >>>> think?
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > > > > > > >>>> issues
> > > > > > > >>>>>>> need
> > > > > > > >>>>>>>>> to be
> > > > > > > >>>>>>>>>>>>> fixed
> > > > > > > >>>>>>>>>>>>>> in
> > > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > > > > > >>>>>>>>>>>>>>>>> Yu
> > > > > > > >>>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> --
> > > > > > > >>>>>>>> Best, Jingsong Lee
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Andrey Zagrebin <az...@apache.org>.
> @Andrey and @Xintong - could we have a quick poll on the user mailing list
> about increasing the metaspace size in Flink 1.10.1? Specifically asking
> for who has very small TM setups?
There has been a survey about this topic since 10 days:
`[Survey] Default size for the new JVM Metaspace limit in 1.10`
I can ask there about the small TM setups.
On Fri, Mar 13, 2020 at 5:19 PM Yu Li <ca...@gmail.com> wrote:
> Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore with
> memory state backends
>
> Let me recompile the watching list with recent feedbacks. There're totally
> 45 issues with Blocker/Critical priority for 1.10.1, out of which 14
> already resolved and 31 left, and the below ones are watched (meaning that
> once the below ones got fixed, we will kick of the releasing with left ones
> postponed to 1.10.2 unless objections):
>
> * Blockers (7)
> - [Under Discussion] FLINK-16018 Improve error reporting when submitting
> batch job (instead of AskTimeoutException)
> - [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
> on repeated job submission
> - [PR Submitted] FLINK-16170 SearchTemplateRequest ClassNotFoundException
> when use flink-sql-connector-elasticsearch7
> - [PR Submitted] FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - [Closed] FLINK-16406 Increase default value for JVM Metaspace to
> minimise its OutOfMemoryError
> - [Open] FLINK-16454 Update the copyright year in NOTICE files
> - [Open] FLINK-16576 State inconsistency on restore with memory state
> backends
>
> * Critical (4)
> - [Open] FLINK-16047 Blink planner produces wrong aggregate results with
> state clean up
> - [PR Submitted] FLINK-16070 Blink planner can not extract correct unique
> key for UpsertStreamTableSink
> - [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
> handled as Fatal Error in TaskManager
> - [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
>
> Please raise your hand if you find any other issues should be put into this
> list, thanks.
>
> Please also note that 1.10.2 version is already created (thanks for the
> help @jincheng), and feel free to adjust/assign fix version to it when
> necessary.
>
> Best Regards,
> Yu
>
>
> On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
>
> > @Andrey and @Xintong - could we have a quick poll on the user mailing
> list
> > about increasing the metaspace size in Flink 1.10.1? Specifically asking
> > for who has very small TM setups?
> >
> > On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
> >
> > > Thanks for the reminder Stephan and the inputs/discussion Andrey and
> > > Xintong. I totally agree that we should try best not to introduce new
> > > issues when resolving the existing one. Will watch closely about the
> > > discussion and conclusion around metaspace configuration.
> > >
> > > From the point as a developer instead of RM, I'm optimistic that
> > reserving
> > > 160MB more memory to JVM metaspace by default won't cause a big
> problem,
> > > based on the assumption that nowadays assigning 2GB/4GB memory per TM
> is
> > > the common case.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com>
> > wrote:
> > >
> > > > I think in FLINK-16406 we have not only increased the default value
> for
> > > > metaspace, but also increased the 'process.size' in default
> > > > 'flink-conf.yaml'.
> > > >
> > > > That means:
> > > >
> > > > - For people using the default 'flink-conf.yaml', it's not really
> > > taking
> > > > memory from heap / managed for metaspace.
> > > > - For people using custom configuration files:
> > > > - If they are specifying 'flink.size', then metaspace does not
> > > really
> > > > affect the flink memory internal components (heap, managed,
> > etc.).
> > > > - If they are specifying 'process.size' (most likely active
> > > setups),
> > > > then they will have less heap / managed memory, or worse, their
> > > > configuration may not work without adjustment.
> > > > - For a TM with large process memory (e.g., 4 GB or more),
> it
> > is
> > > > probably ok that the metaspace takes 160 MB more memory.
> > > > - For a TM with smaller total memory (1 or 2 GB) the
> metaspace
> > > > increase is more likely to cause problem.
> > > >
> > > > So basically only people have small 'process.size' in custom config
> > file
> > > > are really affected. I'm not sure what is the proportion of such use
> > > cases
> > > > though. (From questions asked on the user ML, probably not much).
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org>
> > wrote:
> > > >
> > > > > No need to revert it now - I am not saying it should not go into
> > > 1.10.1,
> > > > I
> > > > > am just saying this is not clear to me yet.
> > > > >
> > > > > We have to trade off the fact that we may break some deployments
> with
> > > the
> > > > > fact that we will "safe" a lot of new deployments.
> > > > > I simply lack the data points / insight at the moment to understand
> > how
> > > > > significant both cases are, meaning how many users would be
> affected
> > > and
> > > > > how badly.
> > > > >
> > > > > Independent of that, improving the error message is always helpful.
> > > > >
> > > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > > > azagrebin.apache@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > > - For 1.10.1 I am not completely sure, because users expect
> to
> > > > > upgrade
> > > > > > > that without config adjustments. That might not be possible
> with
> > > that
> > > > > > > change.
> > > > > >
> > > > > > Ok, makes sense, I will revert it for 1.10 and only try to
> improve
> > > > error
> > > > > > message and docs.
> > > > > >
> > > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org>
> wrote:
> > > > > > >
> > > > > > > @Andrey about the increase in metaspace size
> > > > > > > - I have no concerns for 1.11.0.
> > > > > > > - For 1.10.1 I am not completely sure, because users expect
> to
> > > > > upgrade
> > > > > > > that without config adjustments. That might not be possible
> with
> > > that
> > > > > > > change.
> > > > > > >
> > > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > > > > azagrebin.apache@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >>
> > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > > repeated
> > > > > > >> job”
> > > > > > >>
> > > > > > >> My understanding that the issue is basically covered by:
> > > > > > >>
> > > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> > Fatal
> > > > > Error
> > > > > > >> in TaskManager
> > > > > > >> no full consensus there but improving error message for
> > existing
> > > > > task
> > > > > > >> thread fatal handling could be done at least
> > > > > > >>
> > > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace to
> > > minimise
> > > > > its
> > > > > > >> OutOfMemoryError
> > > > > > >> see further
> > > > > > >>
> > > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
> > dynamically
> > > > > > loaded
> > > > > > >> AWS connectors
> > > > > > >> not sure whether this is a blocker but looks close to be
> > resolved
> > > > > > >>
> > > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > > > >>> - Have we consensus that this is okay for a bugfix release?
> It
> > > > > changes
> > > > > > >>> setups, takes away memory from heap / managed memory on
> > existing
> > > > > setups
> > > > > > >>> that keep their flink-conf.yaml.
> > > > > > >>
> > > > > > >> My understanding was that increasing to 256m resolved the
> > reported
> > > > > > problems
> > > > > > >> and we decided to make the change so I have merged it today as
> > > there
> > > > > > were
> > > > > > >> no more concerns.
> > > > > > >> If there are concerns I can revert it.
> > > > > > >>
> > > > > > >> On the other hand, I think improving the message error with
> > > > reference
> > > > > to
> > > > > > >> the metaspace option should help the most
> > > > > > >> because user would not have to read all docs to fix it
> > > > > > >> then maybe this change is not even needed.
> > > > > > >>
> > > > > > >> Best,
> > > > > > >> Andrey
> > > > > > >>
> > > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org>
> > wrote:
> > > > > > >>>
> > > > > > >>> Good idea to go ahead with 1.10.1
> > > > > > >>>
> > > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > > repeated
> > > > > > >> job"
> > > > > > >>> - I don't think we have consensus on the exact solution, yet,
> > and
> > > > > some
> > > > > > >> of
> > > > > > >>> the changes might also have side effects that are hard to
> > > predict,
> > > > > so I
> > > > > > >> am
> > > > > > >>> not sure we should rush this in.
> > > > > > >>>
> > > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > > > >>> - Have we consensus that this is okay for a bugfix release?
> It
> > > > > changes
> > > > > > >>> setups, takes away memory from heap / managed memory on
> > existing
> > > > > setups
> > > > > > >>> that keep their flink-conf.yaml.
> > > > > > >>>
> > > > > > >>> We may need to unblock the release form these two issues and
> > > think
> > > > > > about
> > > > > > >>> having 1.10.2 in the near future.
> > > > > > >>>
> > > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com>
> > wrote:
> > > > > > >>>
> > > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> > > > > > >>>>
> > > > > > >>>> Best Regards,
> > > > > > >>>> Yu
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com>
> > wrote:
> > > > > > >>>>
> > > > > > >>>>> Thanks for driving this release, Yu!
> > > > > > >>>>> +1 to start 1.10.1 release cycle.
> > > > > > >>>>>
> > > > > > >>>>> From the Table SQL module, I think we should also try to
> get
> > in
> > > > the
> > > > > > >>>>> following issues:
> > > > > > >>>>> - FLINK-16441: Allow users to override flink-conf
> parameters
> > > from
> > > > > SQL
> > > > > > >> CLI
> > > > > > >>>>> environment
> > > > > > >>>>> this allows users to set e.g. statebackend, watermark
> > interval,
> > > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
> > exception
> > > is
> > > > > > >> thrown
> > > > > > >>>>> this will improve much experience when using SQL CLI
> > > > > > >>>>>
> > > > > > >>>>> Best,
> > > > > > >>>>> Jark
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com>
> > wrote:
> > > > > > >>>>>
> > > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> > > > > FLINK-16225
> > > > > > >>>> fix
> > > > > > >>>>>> versions and promoted its priority to Critical. Will also
> > > watch
> > > > > the
> > > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > > > > >>>>>>
> > > > > > >>>>>> Best Regards,
> > > > > > >>>>>> Yu
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > > > > azagrebin@apache.org
> > > > > > >
> > > > > > >>>>>> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>>> Hi Yu,
> > > > > > >>>>>>>
> > > > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > > > > >>>>>>>
> > > > > > >>>>>>> Apart from
> > > > > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to
> > > > > minimise
> > > > > > >>>> its
> > > > > > >>>>>>> OutOfMemoryError
> > > > > > >>>>>>> which should be merged soon
> > > > > > >>>>>>>
> > > > > > >>>>>>> I think we should also try to get in the following
> issues:
> > > > > > >>>>>>>
> > > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled
> > as
> > > > > Fatal
> > > > > > >>>>> Error
> > > > > > >>>>>> in
> > > > > > >>>>>>> TaskManager
> > > > > > >>>>>>> This should solve the Metaspace problem even in a better
> > way
> > > > > > because
> > > > > > >>>>> OOM
> > > > > > >>>>>>> failure should point users to the docs immediately
> > > > > > >>>>>>>
> > > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime
> of
> > a
> > > > slot
> > > > > > >>>>>>> This should give a better protection against class
> loading
> > > > leaks
> > > > > > >>>>>>>
> > > > > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting
> > batch
> > > > job
> > > > > > >>>>>> (instead
> > > > > > >>>>>>> of AskTimeoutException)
> > > > > > >>>>>>> This problem has recently happened for multiple users
> > > > > > >>>>>>>
> > > > > > >>>>>>> Best,
> > > > > > >>>>>>> Andrey
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > > > > jingsonglee0@gmail.com>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
> > release.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Some issues are very important, Users are looking
> forward
> > to
> > > > > them.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Best,
> > > > > > >>>>>>>> Jingsong Lee
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> > > > karmagyz@gmail.com>
> > > > > > >>>>> wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> Best,
> > > > > > >>>>>>>>> Yangze Guo
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > > > > >>>>> tonysong820@gmail.com>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Yu,
> > > > > > >>>>>>>>>> Thanks for the explanation.
> > > > > > >>>>>>>>>> I've no concerns. I was just trying to get some inputs
> > for
> > > > > > >>>>>>> prioritizing
> > > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Thank you~
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Xintong Song
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> > carp84@gmail.com>
> > > > > > >>>> wrote:
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> According to the history, the first patch release of
> a
> > > > major
> > > > > > >>>>>>> version
> > > > > > >>>>>>>>> will
> > > > > > >>>>>>>>>>> take ~1month from discussion started, depending on
> the
> > > > speed
> > > > > > >>>> of
> > > > > > >>>>>>>> blocker
> > > > > > >>>>>>>>>>> issue resolving:
> > > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released
> > on
> > > > Jul
> > > > > > >>>>> 3rd
> > > > > > >>>>>>> [2]
> > > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released
> > on
> > > > Oct
> > > > > > >>>>> 19th
> > > > > > >>>>>>> [4]
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> We won't rush to match the history of course, but
> could
> > > use
> > > > > > >>>> it
> > > > > > >>>>>> as a
> > > > > > >>>>>>>>>>> reference. And please feel free to let me know if any
> > > > > > >>>> concerns
> > > > > > >>>>>>>> Xintong.
> > > > > > >>>>>>>>>>> Thanks.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Best Regards,
> > > > > > >>>>>>>>>>> Yu
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> [1]
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > >>>>>>>>>>> [2]
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > >>>>>>>>>>> [3]
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > >>>>>>>>>>> [4]
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > > > > >>>>>> tonysong820@gmail.com>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be
> the
> > > > > > >>>>> release
> > > > > > >>>>>>>>> manager.
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> +1 for the proposal.
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> One quick question, what is the time plan for
> 1.10.1?
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> Thank you~
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> Xintong Song
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > > > > >>>>>>>>>>>>> +1 on my side
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>> Best,
> > > > > > >>>>>>>>>>>>> Zhijiang
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>
> > > > > ------------------------------------------------------------------
> > > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep
> > an
> > > > > > >>>> eye
> > > > > > >>>>> on
> > > > > > >>>>>>>> these
> > > > > > >>>>>>>>>>>>> existing blocker issues.
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>> Best Regards,
> > > > > > >>>>>>>>>>>>> Yu
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > > > > >>>>> hequn@apache.org>
> > > > > > >>>>>>>>> wrote:
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> Hi Yu,
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
> > volunteer
> > > > > > >>>> as
> > > > > > >>>>>> the
> > > > > > >>>>>>>>> release
> > > > > > >>>>>>>>>>>>>> manager!
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which are
> > > marked
> > > > > > >>>>> as
> > > > > > >>>>>> a
> > > > > > >>>>>>>>> blocker:
> > > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE
> > > files
> > > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
> usrlib
> > > > > > >>>>>> directory
> > > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > > > > >>>>> ClassNotFoundException
> > > > > > >>>>>>> when
> > > > > > >>>>>>>>> use
> > > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
> > submitting
> > > > > > >>>>> batch
> > > > > > >>>>>>> job
> > > > > > >>>>>>>>>>>> (instead
> > > > > > >>>>>>>>>>>>> of
> > > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> Best,
> > > > > > >>>>>>>>>>>>>> Hequn
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> [1]
> > > > > > >>>>>>>>>
> > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > > > > >>>> carp84@gmail.com>
> > > > > > >>>>>>>> wrote:
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a
> lot!
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>> Best Regards,
> > > > > > >>>>>>>>>>>>>>> Yu
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>> wrote:
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would
> > like
> > > > > > >>>>> to
> > > > > > >>>>>>> give
> > > > > > >>>>>>>>> you a
> > > > > > >>>>>>>>>>>>> hand
> > > > > > >>>>>>>>>>>>>> at
> > > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > > > > > >>>> need)
> > > > > > >>>>>> :)
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>> Best,
> > > > > > >>>>>>>>>>>>>>>> Jincheng
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二
> 下午5:49写道：
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> Hi All,
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we released
> > > > > > >>>>> Flink
> > > > > > >>>>>>>>> 1.10.0. We
> > > > > > >>>>>>>>>>>>>> already
> > > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> > > > > > >>>> the
> > > > > > >>>>>>>>>>> release-1.10
> > > > > > >>>>>>>>>>>>>>> branch,
> > > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release
> cycle.
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > > > > > >>>> notice
> > > > > > >>>>>>> file
> > > > > > >>>>>>>> in
> > > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > > > > >>>>>>>>>>>>>>>>> module
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > > > > >>>>>>>>>>>>> flink-state-processor-api
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known
> in
> > > > > > >>>>>>>>> HiveCatalog
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > > > >>>>>>>> TypeExtractor
> > > > > > >>>>>>>>>>> when
> > > > > > >>>>>>>>>>>>>>> mapping
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > > > > >>>> running
> > > > > > >>>>>> in
> > > > > > >>>>>>>>> blink
> > > > > > >>>>>>>>>>>>> planner
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > > > > > >>>> reset
> > > > > > >>>>>> on
> > > > > > >>>>>>>> task
> > > > > > >>>>>>>>>>>>> recovery
> > > > > > >>>>>>>>>>>>>> in
> > > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using
> sql
> > > > > > >>>>>>> casuing
> > > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues
> should
> > > > > > >>>> be
> > > > > > >>>>>>>> merged
> > > > > > >>>>>>>>>>> before
> > > > > > >>>>>>>>>>>>>>> 1.10.1
> > > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> > > > > > >>>>> error
> > > > > > >>>>>> on
> > > > > > >>>>>>>>>>> repeated
> > > > > > >>>>>>>>>>>>> job
> > > > > > >>>>>>>>>>>>>>>>> submission
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > > > > > >>>>>> Metaspace
> > > > > > >>>>>>> to
> > > > > > >>>>>>>>>>>> minimise
> > > > > > >>>>>>>>>>>>>> its
> > > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > > > > >>>>> aggregate
> > > > > > >>>>>>>>> results
> > > > > > >>>>>>>>>>> with
> > > > > > >>>>>>>>>>>>>> state
> > > > > > >>>>>>>>>>>>>>>>> clean up
> > > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > > > > >>>> correct
> > > > > > >>>>>>>> unique
> > > > > > >>>>>>>>> key
> > > > > > >>>>>>>>>>>> for
> > > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and
> kick
> > > > > > >>>>> off
> > > > > > >>>>>>> the
> > > > > > >>>>>>>>>>> release
> > > > > > >>>>>>>>>>>>>>> process
> > > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > > > > >>>> think?
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > > > > > >>>> issues
> > > > > > >>>>>>> need
> > > > > > >>>>>>>>> to be
> > > > > > >>>>>>>>>>>>> fixed
> > > > > > >>>>>>>>>>>>>> in
> > > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > > > > >>>>>>>>>>>>>>>>> Yu
> > > > > > >>>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> --
> > > > > > >>>>>>>> Best, Jingsong Lee
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Another blocker for 1.10.1: FLINK-16576 State inconsistency on restore with
memory state backends
Let me recompile the watching list with recent feedbacks. There're totally
45 issues with Blocker/Critical priority for 1.10.1, out of which 14
already resolved and 31 left, and the below ones are watched (meaning that
once the below ones got fixed, we will kick of the releasing with left ones
postponed to 1.10.2 unless objections):
* Blockers (7)
- [Under Discussion] FLINK-16018 Improve error reporting when submitting
batch job (instead of AskTimeoutException)
- [Under Discussion] FLINK-16142 Memory Leak causes Metaspace OOM error
on repeated job submission
- [PR Submitted] FLINK-16170 SearchTemplateRequest ClassNotFoundException
when use flink-sql-connector-elasticsearch7
- [PR Submitted] FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- [Closed] FLINK-16406 Increase default value for JVM Metaspace to
minimise its OutOfMemoryError
- [Open] FLINK-16454 Update the copyright year in NOTICE files
- [Open] FLINK-16576 State inconsistency on restore with memory state
backends
* Critical (4)
- [Open] FLINK-16047 Blink planner produces wrong aggregate results with
state clean up
- [PR Submitted] FLINK-16070 Blink planner can not extract correct unique
key for UpsertStreamTableSink
- [Under Discussion] FLINK-16225 Metaspace Out Of Memory should be
handled as Fatal Error in TaskManager
- [Open] FLINK-16408 Bind user code class loader to lifetime of a slot
Please raise your hand if you find any other issues should be put into this
list, thanks.
Please also note that 1.10.2 version is already created (thanks for the
help @jincheng), and feel free to adjust/assign fix version to it when
necessary.
Best Regards,
Yu
On Fri, 13 Mar 2020 at 18:21, Stephan Ewen <se...@apache.org> wrote:
> @Andrey and @Xintong - could we have a quick poll on the user mailing list
> about increasing the metaspace size in Flink 1.10.1? Specifically asking
> for who has very small TM setups?
>
> On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
>
> > Thanks for the reminder Stephan and the inputs/discussion Andrey and
> > Xintong. I totally agree that we should try best not to introduce new
> > issues when resolving the existing one. Will watch closely about the
> > discussion and conclusion around metaspace configuration.
> >
> > From the point as a developer instead of RM, I'm optimistic that
> reserving
> > 160MB more memory to JVM metaspace by default won't cause a big problem,
> > based on the assumption that nowadays assigning 2GB/4GB memory per TM is
> > the common case.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com>
> wrote:
> >
> > > I think in FLINK-16406 we have not only increased the default value for
> > > metaspace, but also increased the 'process.size' in default
> > > 'flink-conf.yaml'.
> > >
> > > That means:
> > >
> > > - For people using the default 'flink-conf.yaml', it's not really
> > taking
> > > memory from heap / managed for metaspace.
> > > - For people using custom configuration files:
> > > - If they are specifying 'flink.size', then metaspace does not
> > really
> > > affect the flink memory internal components (heap, managed,
> etc.).
> > > - If they are specifying 'process.size' (most likely active
> > setups),
> > > then they will have less heap / managed memory, or worse, their
> > > configuration may not work without adjustment.
> > > - For a TM with large process memory (e.g., 4 GB or more), it
> is
> > > probably ok that the metaspace takes 160 MB more memory.
> > > - For a TM with smaller total memory (1 or 2 GB) the metaspace
> > > increase is more likely to cause problem.
> > >
> > > So basically only people have small 'process.size' in custom config
> file
> > > are really affected. I'm not sure what is the proportion of such use
> > cases
> > > though. (From questions asked on the user ML, probably not much).
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > >
> > > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org>
> wrote:
> > >
> > > > No need to revert it now - I am not saying it should not go into
> > 1.10.1,
> > > I
> > > > am just saying this is not clear to me yet.
> > > >
> > > > We have to trade off the fact that we may break some deployments with
> > the
> > > > fact that we will "safe" a lot of new deployments.
> > > > I simply lack the data points / insight at the moment to understand
> how
> > > > significant both cases are, meaning how many users would be affected
> > and
> > > > how badly.
> > > >
> > > > Independent of that, improving the error message is always helpful.
> > > >
> > > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > > azagrebin.apache@gmail.com>
> > > > wrote:
> > > >
> > > > > > - For 1.10.1 I am not completely sure, because users expect to
> > > > upgrade
> > > > > > that without config adjustments. That might not be possible with
> > that
> > > > > > change.
> > > > >
> > > > > Ok, makes sense, I will revert it for 1.10 and only try to improve
> > > error
> > > > > message and docs.
> > > > >
> > > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
> > > > > >
> > > > > > @Andrey about the increase in metaspace size
> > > > > > - I have no concerns for 1.11.0.
> > > > > > - For 1.10.1 I am not completely sure, because users expect to
> > > > upgrade
> > > > > > that without config adjustments. That might not be possible with
> > that
> > > > > > change.
> > > > > >
> > > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > > > azagrebin.apache@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >>
> > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > repeated
> > > > > >> job”
> > > > > >>
> > > > > >> My understanding that the issue is basically covered by:
> > > > > >>
> > > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> Fatal
> > > > Error
> > > > > >> in TaskManager
> > > > > >> no full consensus there but improving error message for
> existing
> > > > task
> > > > > >> thread fatal handling could be done at least
> > > > > >>
> > > > > >> - [FLINK-16406] Increase default value for JVM Metaspace to
> > minimise
> > > > its
> > > > > >> OutOfMemoryError
> > > > > >> see further
> > > > > >>
> > > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from
> dynamically
> > > > > loaded
> > > > > >> AWS connectors
> > > > > >> not sure whether this is a blocker but looks close to be
> resolved
> > > > > >>
> > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > > >>> - Have we consensus that this is okay for a bugfix release? It
> > > > changes
> > > > > >>> setups, takes away memory from heap / managed memory on
> existing
> > > > setups
> > > > > >>> that keep their flink-conf.yaml.
> > > > > >>
> > > > > >> My understanding was that increasing to 256m resolved the
> reported
> > > > > problems
> > > > > >> and we decided to make the change so I have merged it today as
> > there
> > > > > were
> > > > > >> no more concerns.
> > > > > >> If there are concerns I can revert it.
> > > > > >>
> > > > > >> On the other hand, I think improving the message error with
> > > reference
> > > > to
> > > > > >> the metaspace option should help the most
> > > > > >> because user would not have to read all docs to fix it
> > > > > >> then maybe this change is not even needed.
> > > > > >>
> > > > > >> Best,
> > > > > >> Andrey
> > > > > >>
> > > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org>
> wrote:
> > > > > >>>
> > > > > >>> Good idea to go ahead with 1.10.1
> > > > > >>>
> > > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > repeated
> > > > > >> job"
> > > > > >>> - I don't think we have consensus on the exact solution, yet,
> and
> > > > some
> > > > > >> of
> > > > > >>> the changes might also have side effects that are hard to
> > predict,
> > > > so I
> > > > > >> am
> > > > > >>> not sure we should rush this in.
> > > > > >>>
> > > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > > >>> - Have we consensus that this is okay for a bugfix release? It
> > > > changes
> > > > > >>> setups, takes away memory from heap / managed memory on
> existing
> > > > setups
> > > > > >>> that keep their flink-conf.yaml.
> > > > > >>>
> > > > > >>> We may need to unblock the release form these two issues and
> > think
> > > > > about
> > > > > >>> having 1.10.2 in the near future.
> > > > > >>>
> > > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com>
> wrote:
> > > > > >>>
> > > > > >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> > > > > >>>>
> > > > > >>>> Best Regards,
> > > > > >>>> Yu
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com>
> wrote:
> > > > > >>>>
> > > > > >>>>> Thanks for driving this release, Yu!
> > > > > >>>>> +1 to start 1.10.1 release cycle.
> > > > > >>>>>
> > > > > >>>>> From the Table SQL module, I think we should also try to get
> in
> > > the
> > > > > >>>>> following issues:
> > > > > >>>>> - FLINK-16441: Allow users to override flink-conf parameters
> > from
> > > > SQL
> > > > > >> CLI
> > > > > >>>>> environment
> > > > > >>>>> this allows users to set e.g. statebackend, watermark
> interval,
> > > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched
> exception
> > is
> > > > > >> thrown
> > > > > >>>>> this will improve much experience when using SQL CLI
> > > > > >>>>>
> > > > > >>>>> Best,
> > > > > >>>>> Jark
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com>
> wrote:
> > > > > >>>>>
> > > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> > > > FLINK-16225
> > > > > >>>> fix
> > > > > >>>>>> versions and promoted its priority to Critical. Will also
> > watch
> > > > the
> > > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > > > >>>>>>
> > > > > >>>>>> Best Regards,
> > > > > >>>>>> Yu
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > > > azagrebin@apache.org
> > > > > >
> > > > > >>>>>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> Hi Yu,
> > > > > >>>>>>>
> > > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > > > >>>>>>>
> > > > > >>>>>>> Apart from
> > > > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to
> > > > minimise
> > > > > >>>> its
> > > > > >>>>>>> OutOfMemoryError
> > > > > >>>>>>> which should be merged soon
> > > > > >>>>>>>
> > > > > >>>>>>> I think we should also try to get in the following issues:
> > > > > >>>>>>>
> > > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled
> as
> > > > Fatal
> > > > > >>>>> Error
> > > > > >>>>>> in
> > > > > >>>>>>> TaskManager
> > > > > >>>>>>> This should solve the Metaspace problem even in a better
> way
> > > > > because
> > > > > >>>>> OOM
> > > > > >>>>>>> failure should point users to the docs immediately
> > > > > >>>>>>>
> > > > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of
> a
> > > slot
> > > > > >>>>>>> This should give a better protection against class loading
> > > leaks
> > > > > >>>>>>>
> > > > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting
> batch
> > > job
> > > > > >>>>>> (instead
> > > > > >>>>>>> of AskTimeoutException)
> > > > > >>>>>>> This problem has recently happened for multiple users
> > > > > >>>>>>>
> > > > > >>>>>>> Best,
> > > > > >>>>>>> Andrey
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > > > jingsonglee0@gmail.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1
> release.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Some issues are very important, Users are looking forward
> to
> > > > them.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Best,
> > > > > >>>>>>>> Jingsong Lee
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> > > karmagyz@gmail.com>
> > > > > >>>>> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Best,
> > > > > >>>>>>>>> Yangze Guo
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > > > >>>>> tonysong820@gmail.com>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Yu,
> > > > > >>>>>>>>>> Thanks for the explanation.
> > > > > >>>>>>>>>> I've no concerns. I was just trying to get some inputs
> for
> > > > > >>>>>>> prioritizing
> > > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Thank you~
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Xintong Song
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <
> carp84@gmail.com>
> > > > > >>>> wrote:
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> According to the history, the first patch release of a
> > > major
> > > > > >>>>>>> version
> > > > > >>>>>>>>> will
> > > > > >>>>>>>>>>> take ~1month from discussion started, depending on the
> > > speed
> > > > > >>>> of
> > > > > >>>>>>>> blocker
> > > > > >>>>>>>>>>> issue resolving:
> > > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released
> on
> > > Jul
> > > > > >>>>> 3rd
> > > > > >>>>>>> [2]
> > > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released
> on
> > > Oct
> > > > > >>>>> 19th
> > > > > >>>>>>> [4]
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> We won't rush to match the history of course, but could
> > use
> > > > > >>>> it
> > > > > >>>>>> as a
> > > > > >>>>>>>>>>> reference. And please feel free to let me know if any
> > > > > >>>> concerns
> > > > > >>>>>>>> Xintong.
> > > > > >>>>>>>>>>> Thanks.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Best Regards,
> > > > > >>>>>>>>>>> Yu
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> [1]
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > >>>>>>>>>>> [2]
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > >>>>>>>>>>> [3]
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > >>>>>>>>>>> [4]
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > > > >>>>>> tonysong820@gmail.com>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> > > > > >>>>> release
> > > > > >>>>>>>>> manager.
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> +1 for the proposal.
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> Thank you~
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> Xintong Song
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > > > >>>>>>>>>>>>> +1 on my side
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Best,
> > > > > >>>>>>>>>>>>> Zhijiang
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>
> > > > ------------------------------------------------------------------
> > > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep
> an
> > > > > >>>> eye
> > > > > >>>>> on
> > > > > >>>>>>>> these
> > > > > >>>>>>>>>>>>> existing blocker issues.
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> Best Regards,
> > > > > >>>>>>>>>>>>> Yu
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > > > >>>>> hequn@apache.org>
> > > > > >>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Hi Yu,
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and
> volunteer
> > > > > >>>> as
> > > > > >>>>>> the
> > > > > >>>>>>>>> release
> > > > > >>>>>>>>>>>>>> manager!
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> I found there are some other issues[1] which are
> > marked
> > > > > >>>>> as
> > > > > >>>>>> a
> > > > > >>>>>>>>> blocker:
> > > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE
> > files
> > > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > > > > >>>>>> directory
> > > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > > > >>>>> ClassNotFoundException
> > > > > >>>>>>> when
> > > > > >>>>>>>>> use
> > > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when
> submitting
> > > > > >>>>> batch
> > > > > >>>>>>> job
> > > > > >>>>>>>>>>>> (instead
> > > > > >>>>>>>>>>>>> of
> > > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> Best,
> > > > > >>>>>>>>>>>>>> Hequn
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> [1]
> > > > > >>>>>>>>>
> > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > > > >>>> carp84@gmail.com>
> > > > > >>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> Best Regards,
> > > > > >>>>>>>>>>>>>>> Yu
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would
> like
> > > > > >>>>> to
> > > > > >>>>>>> give
> > > > > >>>>>>>>> you a
> > > > > >>>>>>>>>>>>> hand
> > > > > >>>>>>>>>>>>>> at
> > > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > > > > >>>> need)
> > > > > >>>>>> :)
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Best,
> > > > > >>>>>>>>>>>>>>>> Jincheng
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> Hi All,
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> It has been almost one month since we released
> > > > > >>>>> Flink
> > > > > >>>>>>>>> 1.10.0. We
> > > > > >>>>>>>>>>>>>> already
> > > > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> > > > > >>>> the
> > > > > >>>>>>>>>>> release-1.10
> > > > > >>>>>>>>>>>>>>> branch,
> > > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > > > > >>>> notice
> > > > > >>>>>>> file
> > > > > >>>>>>>> in
> > > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > > > >>>>>>>>>>>>>>>>> module
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > > > >>>>>>>>>>>>> flink-state-processor-api
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> > > > > >>>>>>>>> HiveCatalog
> > > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > > >>>>>>>> TypeExtractor
> > > > > >>>>>>>>>>> when
> > > > > >>>>>>>>>>>>>>> mapping
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > > > >>>> running
> > > > > >>>>>> in
> > > > > >>>>>>>>> blink
> > > > > >>>>>>>>>>>>> planner
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > > > > >>>> reset
> > > > > >>>>>> on
> > > > > >>>>>>>> task
> > > > > >>>>>>>>>>>>> recovery
> > > > > >>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> > > > > >>>>>>> casuing
> > > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
> > > > > >>>> be
> > > > > >>>>>>>> merged
> > > > > >>>>>>>>>>> before
> > > > > >>>>>>>>>>>>>>> 1.10.1
> > > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> > > > > >>>>> error
> > > > > >>>>>> on
> > > > > >>>>>>>>>>> repeated
> > > > > >>>>>>>>>>>>> job
> > > > > >>>>>>>>>>>>>>>>> submission
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > > > > >>>>>> Metaspace
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>> minimise
> > > > > >>>>>>>>>>>>>> its
> > > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > > > >>>>> aggregate
> > > > > >>>>>>>>> results
> > > > > >>>>>>>>>>> with
> > > > > >>>>>>>>>>>>>> state
> > > > > >>>>>>>>>>>>>>>>> clean up
> > > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > > > >>>> correct
> > > > > >>>>>>>> unique
> > > > > >>>>>>>>> key
> > > > > >>>>>>>>>>>> for
> > > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> > > > > >>>>> off
> > > > > >>>>>>> the
> > > > > >>>>>>>>>>> release
> > > > > >>>>>>>>>>>>>>> process
> > > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > > > >>>> think?
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > > > > >>>> issues
> > > > > >>>>>>> need
> > > > > >>>>>>>>> to be
> > > > > >>>>>>>>>>>>> fixed
> > > > > >>>>>>>>>>>>>> in
> > > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > > > >>>>>>>>>>>>>>>>> Yu
> > > > > >>>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> --
> > > > > >>>>>>>> Best, Jingsong Lee
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>>
> > > > > >>>>
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Stephan Ewen <se...@apache.org>.
@Andrey and @Xintong - could we have a quick poll on the user mailing list
about increasing the metaspace size in Flink 1.10.1? Specifically asking
for who has very small TM setups?
On Fri, Mar 13, 2020 at 6:23 AM Yu Li <ca...@gmail.com> wrote:
> Thanks for the reminder Stephan and the inputs/discussion Andrey and
> Xintong. I totally agree that we should try best not to introduce new
> issues when resolving the existing one. Will watch closely about the
> discussion and conclusion around metaspace configuration.
>
> From the point as a developer instead of RM, I'm optimistic that reserving
> 160MB more memory to JVM metaspace by default won't cause a big problem,
> based on the assumption that nowadays assigning 2GB/4GB memory per TM is
> the common case.
>
> Best Regards,
> Yu
>
>
> On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com> wrote:
>
> > I think in FLINK-16406 we have not only increased the default value for
> > metaspace, but also increased the 'process.size' in default
> > 'flink-conf.yaml'.
> >
> > That means:
> >
> > - For people using the default 'flink-conf.yaml', it's not really
> taking
> > memory from heap / managed for metaspace.
> > - For people using custom configuration files:
> > - If they are specifying 'flink.size', then metaspace does not
> really
> > affect the flink memory internal components (heap, managed, etc.).
> > - If they are specifying 'process.size' (most likely active
> setups),
> > then they will have less heap / managed memory, or worse, their
> > configuration may not work without adjustment.
> > - For a TM with large process memory (e.g., 4 GB or more), it is
> > probably ok that the metaspace takes 160 MB more memory.
> > - For a TM with smaller total memory (1 or 2 GB) the metaspace
> > increase is more likely to cause problem.
> >
> > So basically only people have small 'process.size' in custom config file
> > are really affected. I'm not sure what is the proportion of such use
> cases
> > though. (From questions asked on the user ML, probably not much).
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org> wrote:
> >
> > > No need to revert it now - I am not saying it should not go into
> 1.10.1,
> > I
> > > am just saying this is not clear to me yet.
> > >
> > > We have to trade off the fact that we may break some deployments with
> the
> > > fact that we will "safe" a lot of new deployments.
> > > I simply lack the data points / insight at the moment to understand how
> > > significant both cases are, meaning how many users would be affected
> and
> > > how badly.
> > >
> > > Independent of that, improving the error message is always helpful.
> > >
> > > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > > azagrebin.apache@gmail.com>
> > > wrote:
> > >
> > > > > - For 1.10.1 I am not completely sure, because users expect to
> > > upgrade
> > > > > that without config adjustments. That might not be possible with
> that
> > > > > change.
> > > >
> > > > Ok, makes sense, I will revert it for 1.10 and only try to improve
> > error
> > > > message and docs.
> > > >
> > > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
> > > > >
> > > > > @Andrey about the increase in metaspace size
> > > > > - I have no concerns for 1.11.0.
> > > > > - For 1.10.1 I am not completely sure, because users expect to
> > > upgrade
> > > > > that without config adjustments. That might not be possible with
> that
> > > > > change.
> > > > >
> > > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > > azagrebin.apache@gmail.com>
> > > > > wrote:
> > > > >
> > > > >>
> > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > repeated
> > > > >> job”
> > > > >>
> > > > >> My understanding that the issue is basically covered by:
> > > > >>
> > > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> > > Error
> > > > >> in TaskManager
> > > > >> no full consensus there but improving error message for existing
> > > task
> > > > >> thread fatal handling could be done at least
> > > > >>
> > > > >> - [FLINK-16406] Increase default value for JVM Metaspace to
> minimise
> > > its
> > > > >> OutOfMemoryError
> > > > >> see further
> > > > >>
> > > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically
> > > > loaded
> > > > >> AWS connectors
> > > > >> not sure whether this is a blocker but looks close to be resolved
> > > > >>
> > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > >>> - Have we consensus that this is okay for a bugfix release? It
> > > changes
> > > > >>> setups, takes away memory from heap / managed memory on existing
> > > setups
> > > > >>> that keep their flink-conf.yaml.
> > > > >>
> > > > >> My understanding was that increasing to 256m resolved the reported
> > > > problems
> > > > >> and we decided to make the change so I have merged it today as
> there
> > > > were
> > > > >> no more concerns.
> > > > >> If there are concerns I can revert it.
> > > > >>
> > > > >> On the other hand, I think improving the message error with
> > reference
> > > to
> > > > >> the metaspace option should help the most
> > > > >> because user would not have to read all docs to fix it
> > > > >> then maybe this change is not even needed.
> > > > >>
> > > > >> Best,
> > > > >> Andrey
> > > > >>
> > > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
> > > > >>>
> > > > >>> Good idea to go ahead with 1.10.1
> > > > >>>
> > > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> > repeated
> > > > >> job"
> > > > >>> - I don't think we have consensus on the exact solution, yet, and
> > > some
> > > > >> of
> > > > >>> the changes might also have side effects that are hard to
> predict,
> > > so I
> > > > >> am
> > > > >>> not sure we should rush this in.
> > > > >>>
> > > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > > >>> - Have we consensus that this is okay for a bugfix release? It
> > > changes
> > > > >>> setups, takes away memory from heap / managed memory on existing
> > > setups
> > > > >>> that keep their flink-conf.yaml.
> > > > >>>
> > > > >>> We may need to unblock the release form these two issues and
> think
> > > > about
> > > > >>> having 1.10.2 in the near future.
> > > > >>>
> > > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> > > > >>>
> > > > >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> > > > >>>>
> > > > >>>> Best Regards,
> > > > >>>> Yu
> > > > >>>>
> > > > >>>>
> > > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> > > > >>>>
> > > > >>>>> Thanks for driving this release, Yu!
> > > > >>>>> +1 to start 1.10.1 release cycle.
> > > > >>>>>
> > > > >>>>> From the Table SQL module, I think we should also try to get in
> > the
> > > > >>>>> following issues:
> > > > >>>>> - FLINK-16441: Allow users to override flink-conf parameters
> from
> > > SQL
> > > > >> CLI
> > > > >>>>> environment
> > > > >>>>> this allows users to set e.g. statebackend, watermark interval,
> > > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched exception
> is
> > > > >> thrown
> > > > >>>>> this will improve much experience when using SQL CLI
> > > > >>>>>
> > > > >>>>> Best,
> > > > >>>>> Jark
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> > > > >>>>>
> > > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> > > FLINK-16225
> > > > >>>> fix
> > > > >>>>>> versions and promoted its priority to Critical. Will also
> watch
> > > the
> > > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > > >>>>>>
> > > > >>>>>> Best Regards,
> > > > >>>>>> Yu
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > > azagrebin@apache.org
> > > > >
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Yu,
> > > > >>>>>>>
> > > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > > >>>>>>>
> > > > >>>>>>> Apart from
> > > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to
> > > minimise
> > > > >>>> its
> > > > >>>>>>> OutOfMemoryError
> > > > >>>>>>> which should be merged soon
> > > > >>>>>>>
> > > > >>>>>>> I think we should also try to get in the following issues:
> > > > >>>>>>>
> > > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> > > Fatal
> > > > >>>>> Error
> > > > >>>>>> in
> > > > >>>>>>> TaskManager
> > > > >>>>>>> This should solve the Metaspace problem even in a better way
> > > > because
> > > > >>>>> OOM
> > > > >>>>>>> failure should point users to the docs immediately
> > > > >>>>>>>
> > > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a
> > slot
> > > > >>>>>>> This should give a better protection against class loading
> > leaks
> > > > >>>>>>>
> > > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting batch
> > job
> > > > >>>>>> (instead
> > > > >>>>>>> of AskTimeoutException)
> > > > >>>>>>> This problem has recently happened for multiple users
> > > > >>>>>>>
> > > > >>>>>>> Best,
> > > > >>>>>>> Andrey
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > > jingsonglee0@gmail.com>
> > > > >>>>>>> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > > > >>>>>>>>
> > > > >>>>>>>> Some issues are very important, Users are looking forward to
> > > them.
> > > > >>>>>>>>
> > > > >>>>>>>> Best,
> > > > >>>>>>>> Jingsong Lee
> > > > >>>>>>>>
> > > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> > karmagyz@gmail.com>
> > > > >>>>> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>>> Thanks for driving this release, Yu!
> > > > >>>>>>>>>
> > > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Best,
> > > > >>>>>>>>> Yangze Guo
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > > >>>>> tonysong820@gmail.com>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Yu,
> > > > >>>>>>>>>> Thanks for the explanation.
> > > > >>>>>>>>>> I've no concerns. I was just trying to get some inputs for
> > > > >>>>>>> prioritizing
> > > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Thank you~
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Xintong Song
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> > > > >>>> wrote:
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> According to the history, the first patch release of a
> > major
> > > > >>>>>>> version
> > > > >>>>>>>>> will
> > > > >>>>>>>>>>> take ~1month from discussion started, depending on the
> > speed
> > > > >>>> of
> > > > >>>>>>>> blocker
> > > > >>>>>>>>>>> issue resolving:
> > > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on
> > Jul
> > > > >>>>> 3rd
> > > > >>>>>>> [2]
> > > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on
> > Oct
> > > > >>>>> 19th
> > > > >>>>>>> [4]
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> We won't rush to match the history of course, but could
> use
> > > > >>>> it
> > > > >>>>>> as a
> > > > >>>>>>>>>>> reference. And please feel free to let me know if any
> > > > >>>> concerns
> > > > >>>>>>>> Xintong.
> > > > >>>>>>>>>>> Thanks.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Best Regards,
> > > > >>>>>>>>>>> Yu
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> [1]
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > >>>>>>>>>>> [2]
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > >>>>>>>>>>> [3]
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > >>>>>>>>>>> [4]
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > > >>>>>> tonysong820@gmail.com>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> > > > >>>>> release
> > > > >>>>>>>>> manager.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> +1 for the proposal.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Thank you~
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Xintong Song
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > > >>>>>>>>>>>>> +1 on my side
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Best,
> > > > >>>>>>>>>>>>> Zhijiang
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>
> > > ------------------------------------------------------------------
> > > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
> > > > >>>> eye
> > > > >>>>> on
> > > > >>>>>>>> these
> > > > >>>>>>>>>>>>> existing blocker issues.
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> Best Regards,
> > > > >>>>>>>>>>>>> Yu
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > > >>>>> hequn@apache.org>
> > > > >>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Hi Yu,
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
> > > > >>>> as
> > > > >>>>>> the
> > > > >>>>>>>>> release
> > > > >>>>>>>>>>>>>> manager!
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> I found there are some other issues[1] which are
> marked
> > > > >>>>> as
> > > > >>>>>> a
> > > > >>>>>>>>> blocker:
> > > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE
> files
> > > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > > > >>>>>> directory
> > > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > > >>>>> ClassNotFoundException
> > > > >>>>>>> when
> > > > >>>>>>>>> use
> > > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
> > > > >>>>> batch
> > > > >>>>>>> job
> > > > >>>>>>>>>>>> (instead
> > > > >>>>>>>>>>>>> of
> > > > >>>>>>>>>>>>>> AskTimeoutException)
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> Best,
> > > > >>>>>>>>>>>>>> Hequn
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> [1]
> > > > >>>>>>>>>
> > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > > >>>> carp84@gmail.com>
> > > > >>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> Best Regards,
> > > > >>>>>>>>>>>>>>> Yu
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>> wrote:
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
> > > > >>>>> to
> > > > >>>>>>> give
> > > > >>>>>>>>> you a
> > > > >>>>>>>>>>>>> hand
> > > > >>>>>>>>>>>>>> at
> > > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > > > >>>> need)
> > > > >>>>>> :)
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Best,
> > > > >>>>>>>>>>>>>>>> Jincheng
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Hi All,
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> It has been almost one month since we released
> > > > >>>>> Flink
> > > > >>>>>>>>> 1.10.0. We
> > > > >>>>>>>>>>>>>> already
> > > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> > > > >>>> the
> > > > >>>>>>>>>>> release-1.10
> > > > >>>>>>>>>>>>>>> branch,
> > > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > > > >>>> notice
> > > > >>>>>>> file
> > > > >>>>>>>> in
> > > > >>>>>>>>>>>>>>> flink-ml-lib
> > > > >>>>>>>>>>>>>>>>> module
> > > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > > >>>>>>>>>>>>> flink-state-processor-api
> > > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> > > > >>>>>>>>> HiveCatalog
> > > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > >>>>>>>> TypeExtractor
> > > > >>>>>>>>>>> when
> > > > >>>>>>>>>>>>>>> mapping
> > > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > > >>>> running
> > > > >>>>>> in
> > > > >>>>>>>>> blink
> > > > >>>>>>>>>>>>> planner
> > > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > > > >>>> reset
> > > > >>>>>> on
> > > > >>>>>>>> task
> > > > >>>>>>>>>>>>> recovery
> > > > >>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> > > > >>>>>>> casuing
> > > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
> > > > >>>> be
> > > > >>>>>>>> merged
> > > > >>>>>>>>>>> before
> > > > >>>>>>>>>>>>>>> 1.10.1
> > > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> > > > >>>>> error
> > > > >>>>>> on
> > > > >>>>>>>>>>> repeated
> > > > >>>>>>>>>>>>> job
> > > > >>>>>>>>>>>>>>>>> submission
> > > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > > > >>>>>> Metaspace
> > > > >>>>>>> to
> > > > >>>>>>>>>>>> minimise
> > > > >>>>>>>>>>>>>> its
> > > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > > >>>>> aggregate
> > > > >>>>>>>>> results
> > > > >>>>>>>>>>> with
> > > > >>>>>>>>>>>>>> state
> > > > >>>>>>>>>>>>>>>>> clean up
> > > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > > >>>> correct
> > > > >>>>>>>> unique
> > > > >>>>>>>>> key
> > > > >>>>>>>>>>>> for
> > > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> > > > >>>>> off
> > > > >>>>>>> the
> > > > >>>>>>>>>>> release
> > > > >>>>>>>>>>>>>>> process
> > > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > > >>>> think?
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > > > >>>> issues
> > > > >>>>>>> need
> > > > >>>>>>>>> to be
> > > > >>>>>>>>>>>>> fixed
> > > > >>>>>>>>>>>>>> in
> > > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>> Best Regards,
> > > > >>>>>>>>>>>>>>>>> Yu
> > > > >>>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>>
> > > > >>>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> --
> > > > >>>>>>>> Best, Jingsong Lee
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the reminder Stephan and the inputs/discussion Andrey and
Xintong. I totally agree that we should try best not to introduce new
issues when resolving the existing one. Will watch closely about the
discussion and conclusion around metaspace configuration.
From the point as a developer instead of RM, I'm optimistic that reserving
160MB more memory to JVM metaspace by default won't cause a big problem,
based on the assumption that nowadays assigning 2GB/4GB memory per TM is
the common case.
Best Regards,
Yu
On Fri, 13 Mar 2020 at 09:54, Xintong Song <to...@gmail.com> wrote:
> I think in FLINK-16406 we have not only increased the default value for
> metaspace, but also increased the 'process.size' in default
> 'flink-conf.yaml'.
>
> That means:
>
> - For people using the default 'flink-conf.yaml', it's not really taking
> memory from heap / managed for metaspace.
> - For people using custom configuration files:
> - If they are specifying 'flink.size', then metaspace does not really
> affect the flink memory internal components (heap, managed, etc.).
> - If they are specifying 'process.size' (most likely active setups),
> then they will have less heap / managed memory, or worse, their
> configuration may not work without adjustment.
> - For a TM with large process memory (e.g., 4 GB or more), it is
> probably ok that the metaspace takes 160 MB more memory.
> - For a TM with smaller total memory (1 or 2 GB) the metaspace
> increase is more likely to cause problem.
>
> So basically only people have small 'process.size' in custom config file
> are really affected. I'm not sure what is the proportion of such use cases
> though. (From questions asked on the user ML, probably not much).
>
> Thank you~
>
> Xintong Song
>
>
>
> On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org> wrote:
>
> > No need to revert it now - I am not saying it should not go into 1.10.1,
> I
> > am just saying this is not clear to me yet.
> >
> > We have to trade off the fact that we may break some deployments with the
> > fact that we will "safe" a lot of new deployments.
> > I simply lack the data points / insight at the moment to understand how
> > significant both cases are, meaning how many users would be affected and
> > how badly.
> >
> > Independent of that, improving the error message is always helpful.
> >
> > On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> > azagrebin.apache@gmail.com>
> > wrote:
> >
> > > > - For 1.10.1 I am not completely sure, because users expect to
> > upgrade
> > > > that without config adjustments. That might not be possible with that
> > > > change.
> > >
> > > Ok, makes sense, I will revert it for 1.10 and only try to improve
> error
> > > message and docs.
> > >
> > > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
> > > >
> > > > @Andrey about the increase in metaspace size
> > > > - I have no concerns for 1.11.0.
> > > > - For 1.10.1 I am not completely sure, because users expect to
> > upgrade
> > > > that without config adjustments. That might not be possible with that
> > > > change.
> > > >
> > > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > > azagrebin.apache@gmail.com>
> > > > wrote:
> > > >
> > > >>
> > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> repeated
> > > >> job”
> > > >>
> > > >> My understanding that the issue is basically covered by:
> > > >>
> > > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> > Error
> > > >> in TaskManager
> > > >> no full consensus there but improving error message for existing
> > task
> > > >> thread fatal handling could be done at least
> > > >>
> > > >> - [FLINK-16406] Increase default value for JVM Metaspace to minimise
> > its
> > > >> OutOfMemoryError
> > > >> see further
> > > >>
> > > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically
> > > loaded
> > > >> AWS connectors
> > > >> not sure whether this is a blocker but looks close to be resolved
> > > >>
> > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > >>> - Have we consensus that this is okay for a bugfix release? It
> > changes
> > > >>> setups, takes away memory from heap / managed memory on existing
> > setups
> > > >>> that keep their flink-conf.yaml.
> > > >>
> > > >> My understanding was that increasing to 256m resolved the reported
> > > problems
> > > >> and we decided to make the change so I have merged it today as there
> > > were
> > > >> no more concerns.
> > > >> If there are concerns I can revert it.
> > > >>
> > > >> On the other hand, I think improving the message error with
> reference
> > to
> > > >> the metaspace option should help the most
> > > >> because user would not have to read all docs to fix it
> > > >> then maybe this change is not even needed.
> > > >>
> > > >> Best,
> > > >> Andrey
> > > >>
> > > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
> > > >>>
> > > >>> Good idea to go ahead with 1.10.1
> > > >>>
> > > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on
> repeated
> > > >> job"
> > > >>> - I don't think we have consensus on the exact solution, yet, and
> > some
> > > >> of
> > > >>> the changes might also have side effects that are hard to predict,
> > so I
> > > >> am
> > > >>> not sure we should rush this in.
> > > >>>
> > > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > > >>> - Have we consensus that this is okay for a bugfix release? It
> > changes
> > > >>> setups, takes away memory from heap / managed memory on existing
> > setups
> > > >>> that keep their flink-conf.yaml.
> > > >>>
> > > >>> We may need to unblock the release form these two issues and think
> > > about
> > > >>> having 1.10.2 in the near future.
> > > >>>
> > > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> > > >>>
> > > >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> > > >>>>
> > > >>>> Best Regards,
> > > >>>> Yu
> > > >>>>
> > > >>>>
> > > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> > > >>>>
> > > >>>>> Thanks for driving this release, Yu!
> > > >>>>> +1 to start 1.10.1 release cycle.
> > > >>>>>
> > > >>>>> From the Table SQL module, I think we should also try to get in
> the
> > > >>>>> following issues:
> > > >>>>> - FLINK-16441: Allow users to override flink-conf parameters from
> > SQL
> > > >> CLI
> > > >>>>> environment
> > > >>>>> this allows users to set e.g. statebackend, watermark interval,
> > > >>>>> exactly-once/at-least-once, in the SQL CLI
> > > >>>>> - FLINK-16217: SQL Client crashed when any uncatched exception is
> > > >> thrown
> > > >>>>> this will improve much experience when using SQL CLI
> > > >>>>>
> > > >>>>> Best,
> > > >>>>> Jark
> > > >>>>>
> > > >>>>>
> > > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> > > >>>>>
> > > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> > FLINK-16225
> > > >>>> fix
> > > >>>>>> versions and promoted its priority to Critical. Will also watch
> > the
> > > >>>>>> progress of FLINK-16108/FLINK-16408.
> > > >>>>>>
> > > >>>>>> Best Regards,
> > > >>>>>> Yu
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> > azagrebin@apache.org
> > > >
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Yu,
> > > >>>>>>>
> > > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > > >>>>>>>
> > > >>>>>>> Apart from
> > > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to
> > minimise
> > > >>>> its
> > > >>>>>>> OutOfMemoryError
> > > >>>>>>> which should be merged soon
> > > >>>>>>>
> > > >>>>>>> I think we should also try to get in the following issues:
> > > >>>>>>>
> > > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> > Fatal
> > > >>>>> Error
> > > >>>>>> in
> > > >>>>>>> TaskManager
> > > >>>>>>> This should solve the Metaspace problem even in a better way
> > > because
> > > >>>>> OOM
> > > >>>>>>> failure should point users to the docs immediately
> > > >>>>>>>
> > > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a
> slot
> > > >>>>>>> This should give a better protection against class loading
> leaks
> > > >>>>>>>
> > > >>>>>>> - [FLINK-16018] Improve error reporting when submitting batch
> job
> > > >>>>>> (instead
> > > >>>>>>> of AskTimeoutException)
> > > >>>>>>> This problem has recently happened for multiple users
> > > >>>>>>>
> > > >>>>>>> Best,
> > > >>>>>>> Andrey
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > > jingsonglee0@gmail.com>
> > > >>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > > >>>>>>>>
> > > >>>>>>>> Some issues are very important, Users are looking forward to
> > them.
> > > >>>>>>>>
> > > >>>>>>>> Best,
> > > >>>>>>>> Jingsong Lee
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <
> karmagyz@gmail.com>
> > > >>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Thanks for driving this release, Yu!
> > > >>>>>>>>>
> > > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > > >>>>>>>>>
> > > >>>>>>>>> Best,
> > > >>>>>>>>> Yangze Guo
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > > >>>>> tonysong820@gmail.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yu,
> > > >>>>>>>>>> Thanks for the explanation.
> > > >>>>>>>>>> I've no concerns. I was just trying to get some inputs for
> > > >>>>>>> prioritizing
> > > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thank you~
> > > >>>>>>>>>>
> > > >>>>>>>>>> Xintong Song
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> > > >>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> According to the history, the first patch release of a
> major
> > > >>>>>>> version
> > > >>>>>>>>> will
> > > >>>>>>>>>>> take ~1month from discussion started, depending on the
> speed
> > > >>>> of
> > > >>>>>>>> blocker
> > > >>>>>>>>>>> issue resolving:
> > > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on
> Jul
> > > >>>>> 3rd
> > > >>>>>>> [2]
> > > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on
> Oct
> > > >>>>> 19th
> > > >>>>>>> [4]
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> We won't rush to match the history of course, but could use
> > > >>>> it
> > > >>>>>> as a
> > > >>>>>>>>>>> reference. And please feel free to let me know if any
> > > >>>> concerns
> > > >>>>>>>> Xintong.
> > > >>>>>>>>>>> Thanks.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>> Yu
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> [1]
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > >>>>>>>>>>> [2]
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > >>>>>>>>>>> [3]
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > >>>>>>>>>>> [4]
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > >>>>>> tonysong820@gmail.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> > > >>>>> release
> > > >>>>>>>>> manager.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> +1 for the proposal.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Thank you~
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Xintong Song
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > > >>>>>>>>>>>>> +1 on my side
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Best,
> > > >>>>>>>>>>>>> Zhijiang
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>
> > ------------------------------------------------------------------
> > > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
> > > >>>> eye
> > > >>>>> on
> > > >>>>>>>> these
> > > >>>>>>>>>>>>> existing blocker issues.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>>>> Yu
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > > >>>>> hequn@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Hi Yu,
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
> > > >>>> as
> > > >>>>>> the
> > > >>>>>>>>> release
> > > >>>>>>>>>>>>>> manager!
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> I found there are some other issues[1] which are marked
> > > >>>>> as
> > > >>>>>> a
> > > >>>>>>>>> blocker:
> > > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
> > > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > > >>>>>> directory
> > > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > > >>>>> ClassNotFoundException
> > > >>>>>>> when
> > > >>>>>>>>> use
> > > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
> > > >>>>> batch
> > > >>>>>>> job
> > > >>>>>>>>>>>> (instead
> > > >>>>>>>>>>>>> of
> > > >>>>>>>>>>>>>> AskTimeoutException)
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Best,
> > > >>>>>>>>>>>>>> Hequn
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> [1]
> > > >>>>>>>>>
> > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > > >>>> carp84@gmail.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Hi Jincheng,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>>>>>> Yu
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > >>>>>>>>>>> sunjincheng121@gmail.com
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
> > > >>>>> to
> > > >>>>>>> give
> > > >>>>>>>>> you a
> > > >>>>>>>>>>>>> hand
> > > >>>>>>>>>>>>>> at
> > > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > > >>>> need)
> > > >>>>>> :)
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Best,
> > > >>>>>>>>>>>>>>>> Jincheng
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Hi All,
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> It has been almost one month since we released
> > > >>>>> Flink
> > > >>>>>>>>> 1.10.0. We
> > > >>>>>>>>>>>>>> already
> > > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> > > >>>> the
> > > >>>>>>>>>>> release-1.10
> > > >>>>>>>>>>>>>>> branch,
> > > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > > >>>> notice
> > > >>>>>>> file
> > > >>>>>>>> in
> > > >>>>>>>>>>>>>>> flink-ml-lib
> > > >>>>>>>>>>>>>>>>> module
> > > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > > >>>>>>>>>>>>> flink-state-processor-api
> > > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> > > >>>>>>>>> HiveCatalog
> > > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > > >>>>>>>> TypeExtractor
> > > >>>>>>>>>>> when
> > > >>>>>>>>>>>>>>> mapping
> > > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > > >>>> running
> > > >>>>>> in
> > > >>>>>>>>> blink
> > > >>>>>>>>>>>>> planner
> > > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > > >>>> reset
> > > >>>>>> on
> > > >>>>>>>> task
> > > >>>>>>>>>>>>> recovery
> > > >>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>> DefaultScheduler
> > > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> > > >>>>>>> casuing
> > > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
> > > >>>> be
> > > >>>>>>>> merged
> > > >>>>>>>>>>> before
> > > >>>>>>>>>>>>>>> 1.10.1
> > > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> > > >>>>> error
> > > >>>>>> on
> > > >>>>>>>>>>> repeated
> > > >>>>>>>>>>>>> job
> > > >>>>>>>>>>>>>>>>> submission
> > > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > > >>>>>> Metaspace
> > > >>>>>>> to
> > > >>>>>>>>>>>> minimise
> > > >>>>>>>>>>>>>> its
> > > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > > >>>>> aggregate
> > > >>>>>>>>> results
> > > >>>>>>>>>>> with
> > > >>>>>>>>>>>>>> state
> > > >>>>>>>>>>>>>>>>> clean up
> > > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > > >>>> correct
> > > >>>>>>>> unique
> > > >>>>>>>>> key
> > > >>>>>>>>>>>> for
> > > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> > > >>>>> off
> > > >>>>>>> the
> > > >>>>>>>>>>> release
> > > >>>>>>>>>>>>>>> process
> > > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > > >>>> think?
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > > >>>> issues
> > > >>>>>>> need
> > > >>>>>>>>> to be
> > > >>>>>>>>>>>>> fixed
> > > >>>>>>>>>>>>>> in
> > > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Best Regards,
> > > >>>>>>>>>>>>>>>>> Yu
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Best, Jingsong Lee
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > > >>
> > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Xintong Song <to...@gmail.com>.
I think in FLINK-16406 we have not only increased the default value for
metaspace, but also increased the 'process.size' in default
'flink-conf.yaml'.
That means:
- For people using the default 'flink-conf.yaml', it's not really taking
memory from heap / managed for metaspace.
- For people using custom configuration files:
- If they are specifying 'flink.size', then metaspace does not really
affect the flink memory internal components (heap, managed, etc.).
- If they are specifying 'process.size' (most likely active setups),
then they will have less heap / managed memory, or worse, their
configuration may not work without adjustment.
- For a TM with large process memory (e.g., 4 GB or more), it is
probably ok that the metaspace takes 160 MB more memory.
- For a TM with smaller total memory (1 or 2 GB) the metaspace
increase is more likely to cause problem.
So basically only people have small 'process.size' in custom config file
are really affected. I'm not sure what is the proportion of such use cases
though. (From questions asked on the user ML, probably not much).
Thank you~
Xintong Song
On Thu, Mar 12, 2020 at 10:09 PM Stephan Ewen <se...@apache.org> wrote:
> No need to revert it now - I am not saying it should not go into 1.10.1, I
> am just saying this is not clear to me yet.
>
> We have to trade off the fact that we may break some deployments with the
> fact that we will "safe" a lot of new deployments.
> I simply lack the data points / insight at the moment to understand how
> significant both cases are, meaning how many users would be affected and
> how badly.
>
> Independent of that, improving the error message is always helpful.
>
> On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <
> azagrebin.apache@gmail.com>
> wrote:
>
> > > - For 1.10.1 I am not completely sure, because users expect to
> upgrade
> > > that without config adjustments. That might not be possible with that
> > > change.
> >
> > Ok, makes sense, I will revert it for 1.10 and only try to improve error
> > message and docs.
> >
> > > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
> > >
> > > @Andrey about the increase in metaspace size
> > > - I have no concerns for 1.11.0.
> > > - For 1.10.1 I am not completely sure, because users expect to
> upgrade
> > > that without config adjustments. That might not be possible with that
> > > change.
> > >
> > > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> > azagrebin.apache@gmail.com>
> > > wrote:
> > >
> > >>
> > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> > >> job”
> > >>
> > >> My understanding that the issue is basically covered by:
> > >>
> > >> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> Error
> > >> in TaskManager
> > >> no full consensus there but improving error message for existing
> task
> > >> thread fatal handling could be done at least
> > >>
> > >> - [FLINK-16406] Increase default value for JVM Metaspace to minimise
> its
> > >> OutOfMemoryError
> > >> see further
> > >>
> > >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically
> > loaded
> > >> AWS connectors
> > >> not sure whether this is a blocker but looks close to be resolved
> > >>
> > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > >>> - Have we consensus that this is okay for a bugfix release? It
> changes
> > >>> setups, takes away memory from heap / managed memory on existing
> setups
> > >>> that keep their flink-conf.yaml.
> > >>
> > >> My understanding was that increasing to 256m resolved the reported
> > problems
> > >> and we decided to make the change so I have merged it today as there
> > were
> > >> no more concerns.
> > >> If there are concerns I can revert it.
> > >>
> > >> On the other hand, I think improving the message error with reference
> to
> > >> the metaspace option should help the most
> > >> because user would not have to read all docs to fix it
> > >> then maybe this change is not even needed.
> > >>
> > >> Best,
> > >> Andrey
> > >>
> > >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
> > >>>
> > >>> Good idea to go ahead with 1.10.1
> > >>>
> > >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> > >> job"
> > >>> - I don't think we have consensus on the exact solution, yet, and
> some
> > >> of
> > >>> the changes might also have side effects that are hard to predict,
> so I
> > >> am
> > >>> not sure we should rush this in.
> > >>>
> > >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> > >>> - Have we consensus that this is okay for a bugfix release? It
> changes
> > >>> setups, takes away memory from heap / managed memory on existing
> setups
> > >>> that keep their flink-conf.yaml.
> > >>>
> > >>> We may need to unblock the release form these two issues and think
> > about
> > >>> having 1.10.2 in the near future.
> > >>>
> > >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> > >>>
> > >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> > >>>>
> > >>>> Best Regards,
> > >>>> Yu
> > >>>>
> > >>>>
> > >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> > >>>>
> > >>>>> Thanks for driving this release, Yu!
> > >>>>> +1 to start 1.10.1 release cycle.
> > >>>>>
> > >>>>> From the Table SQL module, I think we should also try to get in the
> > >>>>> following issues:
> > >>>>> - FLINK-16441: Allow users to override flink-conf parameters from
> SQL
> > >> CLI
> > >>>>> environment
> > >>>>> this allows users to set e.g. statebackend, watermark interval,
> > >>>>> exactly-once/at-least-once, in the SQL CLI
> > >>>>> - FLINK-16217: SQL Client crashed when any uncatched exception is
> > >> thrown
> > >>>>> this will improve much experience when using SQL CLI
> > >>>>>
> > >>>>> Best,
> > >>>>> Jark
> > >>>>>
> > >>>>>
> > >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> > >>>>>
> > >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into
> FLINK-16225
> > >>>> fix
> > >>>>>> versions and promoted its priority to Critical. Will also watch
> the
> > >>>>>> progress of FLINK-16108/FLINK-16408.
> > >>>>>>
> > >>>>>> Best Regards,
> > >>>>>> Yu
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <
> azagrebin@apache.org
> > >
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Yu,
> > >>>>>>>
> > >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> > >>>>>>>
> > >>>>>>> Apart from
> > >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to
> minimise
> > >>>> its
> > >>>>>>> OutOfMemoryError
> > >>>>>>> which should be merged soon
> > >>>>>>>
> > >>>>>>> I think we should also try to get in the following issues:
> > >>>>>>>
> > >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as
> Fatal
> > >>>>> Error
> > >>>>>> in
> > >>>>>>> TaskManager
> > >>>>>>> This should solve the Metaspace problem even in a better way
> > because
> > >>>>> OOM
> > >>>>>>> failure should point users to the docs immediately
> > >>>>>>>
> > >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a slot
> > >>>>>>> This should give a better protection against class loading leaks
> > >>>>>>>
> > >>>>>>> - [FLINK-16018] Improve error reporting when submitting batch job
> > >>>>>> (instead
> > >>>>>>> of AskTimeoutException)
> > >>>>>>> This problem has recently happened for multiple users
> > >>>>>>>
> > >>>>>>> Best,
> > >>>>>>> Andrey
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> > jingsonglee0@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > >>>>>>>>
> > >>>>>>>> Some issues are very important, Users are looking forward to
> them.
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>> Jingsong Lee
> > >>>>>>>>
> > >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
> > >>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>
> > >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> > >>>>>>>>>
> > >>>>>>>>> Best,
> > >>>>>>>>> Yangze Guo
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > >>>>> tonysong820@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Yu,
> > >>>>>>>>>> Thanks for the explanation.
> > >>>>>>>>>> I've no concerns. I was just trying to get some inputs for
> > >>>>>>> prioritizing
> > >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Thank you~
> > >>>>>>>>>>
> > >>>>>>>>>> Xintong Song
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> > >>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> > >>>>>>>>>>>
> > >>>>>>>>>>> According to the history, the first patch release of a major
> > >>>>>>> version
> > >>>>>>>>> will
> > >>>>>>>>>>> take ~1month from discussion started, depending on the speed
> > >>>> of
> > >>>>>>>> blocker
> > >>>>>>>>>>> issue resolving:
> > >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on Jul
> > >>>>> 3rd
> > >>>>>>> [2]
> > >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on Oct
> > >>>>> 19th
> > >>>>>>> [4]
> > >>>>>>>>>>>
> > >>>>>>>>>>> We won't rush to match the history of course, but could use
> > >>>> it
> > >>>>>> as a
> > >>>>>>>>>>> reference. And please feel free to let me know if any
> > >>>> concerns
> > >>>>>>>> Xintong.
> > >>>>>>>>>>> Thanks.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Best Regards,
> > >>>>>>>>>>> Yu
> > >>>>>>>>>>>
> > >>>>>>>>>>> [1]
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > >>>>>>>>>>> [2]
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > >>>>>>>>>>> [3]
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > >>>>>>>>>>> [4]
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > >>>>>> tonysong820@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> > >>>>> release
> > >>>>>>>>> manager.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> +1 for the proposal.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thank you~
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Xintong Song
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for driving this release, Yu!
> > >>>>>>>>>>>>> +1 on my side
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>> Zhijiang
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>
> ------------------------------------------------------------------
> > >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> > >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> > >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> > >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
> > >>>> eye
> > >>>>> on
> > >>>>>>>> these
> > >>>>>>>>>>>>> existing blocker issues.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > >>>>> hequn@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi Yu,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
> > >>>> as
> > >>>>>> the
> > >>>>>>>>> release
> > >>>>>>>>>>>>>> manager!
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I found there are some other issues[1] which are marked
> > >>>>> as
> > >>>>>> a
> > >>>>>>>>> blocker:
> > >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
> > >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> > >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > >>>>>> directory
> > >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> > >>>>> ClassNotFoundException
> > >>>>>>> when
> > >>>>>>>>> use
> > >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> > >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
> > >>>>> batch
> > >>>>>>> job
> > >>>>>>>>>>>> (instead
> > >>>>>>>>>>>>> of
> > >>>>>>>>>>>>>> AskTimeoutException)
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>> Hequn
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> [1]
> > >>>>>>>>>
> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> > >>>> carp84@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Jincheng,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > >>>>>>>>>>> sunjincheng121@gmail.com
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
> > >>>>> to
> > >>>>>>> give
> > >>>>>>>>> you a
> > >>>>>>>>>>>>> hand
> > >>>>>>>>>>>>>> at
> > >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> > >>>> need)
> > >>>>>> :)
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>>>> Jincheng
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hi All,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> It has been almost one month since we released
> > >>>>> Flink
> > >>>>>>>>> 1.10.0. We
> > >>>>>>>>>>>>>> already
> > >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> > >>>> the
> > >>>>>>>>>>> release-1.10
> > >>>>>>>>>>>>>>> branch,
> > >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> > >>>> notice
> > >>>>>>> file
> > >>>>>>>> in
> > >>>>>>>>>>>>>>> flink-ml-lib
> > >>>>>>>>>>>>>>>>> module
> > >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> > >>>>>>>>>>>>> flink-state-processor-api
> > >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> > >>>>>>>>> HiveCatalog
> > >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> > >>>>>>>> TypeExtractor
> > >>>>>>>>>>> when
> > >>>>>>>>>>>>>>> mapping
> > >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> > >>>> running
> > >>>>>> in
> > >>>>>>>>> blink
> > >>>>>>>>>>>>> planner
> > >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> > >>>> reset
> > >>>>>> on
> > >>>>>>>> task
> > >>>>>>>>>>>>> recovery
> > >>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>> DefaultScheduler
> > >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> > >>>>>>> casuing
> > >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
> > >>>> be
> > >>>>>>>> merged
> > >>>>>>>>>>> before
> > >>>>>>>>>>>>>>> 1.10.1
> > >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> > >>>>> error
> > >>>>>> on
> > >>>>>>>>>>> repeated
> > >>>>>>>>>>>>> job
> > >>>>>>>>>>>>>>>>> submission
> > >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> > >>>>>> Metaspace
> > >>>>>>> to
> > >>>>>>>>>>>> minimise
> > >>>>>>>>>>>>>> its
> > >>>>>>>>>>>>>>>>> OutOfMemoryError
> > >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> > >>>>> aggregate
> > >>>>>>>>> results
> > >>>>>>>>>>> with
> > >>>>>>>>>>>>>> state
> > >>>>>>>>>>>>>>>>> clean up
> > >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> > >>>> correct
> > >>>>>>>> unique
> > >>>>>>>>> key
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> > >>>>> off
> > >>>>>>> the
> > >>>>>>>>>>> release
> > >>>>>>>>>>>>>>> process
> > >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> > >>>> think?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> > >>>> issues
> > >>>>>>> need
> > >>>>>>>>> to be
> > >>>>>>>>>>>>> fixed
> > >>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>> Yu
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Best, Jingsong Lee
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Stephan Ewen <se...@apache.org>.
No need to revert it now - I am not saying it should not go into 1.10.1, I
am just saying this is not clear to me yet.
We have to trade off the fact that we may break some deployments with the
fact that we will "safe" a lot of new deployments.
I simply lack the data points / insight at the moment to understand how
significant both cases are, meaning how many users would be affected and
how badly.
Independent of that, improving the error message is always helpful.
On Thu, Mar 12, 2020 at 1:22 PM Andrey Zagrebin <az...@gmail.com>
wrote:
> > - For 1.10.1 I am not completely sure, because users expect to upgrade
> > that without config adjustments. That might not be possible with that
> > change.
>
> Ok, makes sense, I will revert it for 1.10 and only try to improve error
> message and docs.
>
> > On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
> >
> > @Andrey about the increase in metaspace size
> > - I have no concerns for 1.11.0.
> > - For 1.10.1 I am not completely sure, because users expect to upgrade
> > that without config adjustments. That might not be possible with that
> > change.
> >
> > On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <
> azagrebin.apache@gmail.com>
> > wrote:
> >
> >>
> >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> >> job”
> >>
> >> My understanding that the issue is basically covered by:
> >>
> >> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error
> >> in TaskManager
> >> no full consensus there but improving error message for existing task
> >> thread fatal handling could be done at least
> >>
> >> - [FLINK-16406] Increase default value for JVM Metaspace to minimise its
> >> OutOfMemoryError
> >> see further
> >>
> >> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically
> loaded
> >> AWS connectors
> >> not sure whether this is a blocker but looks close to be resolved
> >>
> >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> >>> - Have we consensus that this is okay for a bugfix release? It changes
> >>> setups, takes away memory from heap / managed memory on existing setups
> >>> that keep their flink-conf.yaml.
> >>
> >> My understanding was that increasing to 256m resolved the reported
> problems
> >> and we decided to make the change so I have merged it today as there
> were
> >> no more concerns.
> >> If there are concerns I can revert it.
> >>
> >> On the other hand, I think improving the message error with reference to
> >> the metaspace option should help the most
> >> because user would not have to read all docs to fix it
> >> then maybe this change is not even needed.
> >>
> >> Best,
> >> Andrey
> >>
> >>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
> >>>
> >>> Good idea to go ahead with 1.10.1
> >>>
> >>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> >> job"
> >>> - I don't think we have consensus on the exact solution, yet, and some
> >> of
> >>> the changes might also have side effects that are hard to predict, so I
> >> am
> >>> not sure we should rush this in.
> >>>
> >>> About "FLINK-16406 Increase default value for JVM Metaspace"
> >>> - Have we consensus that this is okay for a bugfix release? It changes
> >>> setups, takes away memory from heap / managed memory on existing setups
> >>> that keep their flink-conf.yaml.
> >>>
> >>> We may need to unblock the release form these two issues and think
> about
> >>> having 1.10.2 in the near future.
> >>>
> >>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> >>>
> >>>> Thanks for the reminder Jark. Will keep an eye on these two.
> >>>>
> >>>> Best Regards,
> >>>> Yu
> >>>>
> >>>>
> >>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> >>>>
> >>>>> Thanks for driving this release, Yu!
> >>>>> +1 to start 1.10.1 release cycle.
> >>>>>
> >>>>> From the Table SQL module, I think we should also try to get in the
> >>>>> following issues:
> >>>>> - FLINK-16441: Allow users to override flink-conf parameters from SQL
> >> CLI
> >>>>> environment
> >>>>> this allows users to set e.g. statebackend, watermark interval,
> >>>>> exactly-once/at-least-once, in the SQL CLI
> >>>>> - FLINK-16217: SQL Client crashed when any uncatched exception is
> >> thrown
> >>>>> this will improve much experience when using SQL CLI
> >>>>>
> >>>>> Best,
> >>>>> Jark
> >>>>>
> >>>>>
> >>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> >>>>>
> >>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
> >>>> fix
> >>>>>> versions and promoted its priority to Critical. Will also watch the
> >>>>>> progress of FLINK-16108/FLINK-16408.
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Yu
> >>>>>>
> >>>>>>
> >>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <azagrebin@apache.org
> >
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Yu,
> >>>>>>>
> >>>>>>> Thanks for kicking off the 1.10.1 release discussion!
> >>>>>>>
> >>>>>>> Apart from
> >>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to minimise
> >>>> its
> >>>>>>> OutOfMemoryError
> >>>>>>> which should be merged soon
> >>>>>>>
> >>>>>>> I think we should also try to get in the following issues:
> >>>>>>>
> >>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> >>>>> Error
> >>>>>> in
> >>>>>>> TaskManager
> >>>>>>> This should solve the Metaspace problem even in a better way
> because
> >>>>> OOM
> >>>>>>> failure should point users to the docs immediately
> >>>>>>>
> >>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a slot
> >>>>>>> This should give a better protection against class loading leaks
> >>>>>>>
> >>>>>>> - [FLINK-16018] Improve error reporting when submitting batch job
> >>>>>> (instead
> >>>>>>> of AskTimeoutException)
> >>>>>>> This problem has recently happened for multiple users
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Andrey
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <
> jingsonglee0@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> >>>>>>>>
> >>>>>>>> Some issues are very important, Users are looking forward to them.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Jingsong Lee
> >>>>>>>>
> >>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>
> >>>>>>>>> +1 for starting the 1.10.1 release cycle.
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Yangze Guo
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> >>>>> tonysong820@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Yu,
> >>>>>>>>>> Thanks for the explanation.
> >>>>>>>>>> I've no concerns. I was just trying to get some inputs for
> >>>>>>> prioritizing
> >>>>>>>>>> tasks on my side, and ~1month sounds good to me.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thank you~
> >>>>>>>>>>
> >>>>>>>>>> Xintong Song
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> >>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> bq. what is the time plan for 1.10.1?
> >>>>>>>>>>>
> >>>>>>>>>>> According to the history, the first patch release of a major
> >>>>>>> version
> >>>>>>>>> will
> >>>>>>>>>>> take ~1month from discussion started, depending on the speed
> >>>> of
> >>>>>>>> blocker
> >>>>>>>>>>> issue resolving:
> >>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on Jul
> >>>>> 3rd
> >>>>>>> [2]
> >>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on Oct
> >>>>> 19th
> >>>>>>> [4]
> >>>>>>>>>>>
> >>>>>>>>>>> We won't rush to match the history of course, but could use
> >>>> it
> >>>>>> as a
> >>>>>>>>>>> reference. And please feel free to let me know if any
> >>>> concerns
> >>>>>>>> Xintong.
> >>>>>>>>>>> Thanks.
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Yu
> >>>>>>>>>>>
> >>>>>>>>>>> [1]
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >>>>>>>>>>> [2]
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >>>>>>>>>>> [3]
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >>>>>>>>>>> [4]
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> >>>>>> tonysong820@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> >>>>> release
> >>>>>>>>> manager.
> >>>>>>>>>>>>
> >>>>>>>>>>>> +1 for the proposal.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thank you~
> >>>>>>>>>>>>
> >>>>>>>>>>>> Xintong Song
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> >>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>>>> +1 on my side
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best,
> >>>>>>>>>>>>> Zhijiang
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>> ------------------------------------------------------------------
> >>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >>>>>>>>>>>>> To:dev <de...@flink.apache.org>
> >>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
> >>>> eye
> >>>>> on
> >>>>>>>> these
> >>>>>>>>>>>>> existing blocker issues.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> >>>>> hequn@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
> >>>> as
> >>>>>> the
> >>>>>>>>> release
> >>>>>>>>>>>>>> manager!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I found there are some other issues[1] which are marked
> >>>>> as
> >>>>>> a
> >>>>>>>>> blocker:
> >>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
> >>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
> >>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> >>>>>> directory
> >>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> >>>>> ClassNotFoundException
> >>>>>>> when
> >>>>>>>>> use
> >>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
> >>>>> batch
> >>>>>>> job
> >>>>>>>>>>>> (instead
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>> AskTimeoutException)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>> Hequn
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1]
> >>>>>>>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> >>>> carp84@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Jincheng,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> >>>>>>>>>>> sunjincheng121@gmail.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
> >>>>> to
> >>>>>>> give
> >>>>>>>>> you a
> >>>>>>>>>>>>> hand
> >>>>>>>>>>>>>> at
> >>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> >>>> need)
> >>>>>> :)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>>> Jincheng
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It has been almost one month since we released
> >>>>> Flink
> >>>>>>>>> 1.10.0. We
> >>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> >>>> the
> >>>>>>>>>>> release-1.10
> >>>>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Most noticeable fixes are:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> >>>> notice
> >>>>>>> file
> >>>>>>>> in
> >>>>>>>>>>>>>>> flink-ml-lib
> >>>>>>>>>>>>>>>>> module
> >>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> >>>>>>>>>>>>> flink-state-processor-api
> >>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> >>>>>>>>> HiveCatalog
> >>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> >>>>>>>> TypeExtractor
> >>>>>>>>>>> when
> >>>>>>>>>>>>>>> mapping
> >>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> >>>> running
> >>>>>> in
> >>>>>>>>> blink
> >>>>>>>>>>>>> planner
> >>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> >>>> reset
> >>>>>> on
> >>>>>>>> task
> >>>>>>>>>>>>> recovery
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> DefaultScheduler
> >>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> >>>>>>> casuing
> >>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
> >>>> be
> >>>>>>>> merged
> >>>>>>>>>>> before
> >>>>>>>>>>>>>>> 1.10.1
> >>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> >>>>> error
> >>>>>> on
> >>>>>>>>>>> repeated
> >>>>>>>>>>>>> job
> >>>>>>>>>>>>>>>>> submission
> >>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> >>>>>> Metaspace
> >>>>>>> to
> >>>>>>>>>>>> minimise
> >>>>>>>>>>>>>> its
> >>>>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> >>>>> aggregate
> >>>>>>>>> results
> >>>>>>>>>>> with
> >>>>>>>>>>>>>> state
> >>>>>>>>>>>>>>>>> clean up
> >>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> >>>> correct
> >>>>>>>> unique
> >>>>>>>>> key
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>> UpsertStreamTableSink
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> >>>>> off
> >>>>>>> the
> >>>>>>>>>>> release
> >>>>>>>>>>>>>>> process
> >>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
> >>>> think?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
> >>>> issues
> >>>>>>> need
> >>>>>>>>> to be
> >>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Best, Jingsong Lee
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Andrey Zagrebin <az...@gmail.com>.
> - For 1.10.1 I am not completely sure, because users expect to upgrade
> that without config adjustments. That might not be possible with that
> change.
Ok, makes sense, I will revert it for 1.10 and only try to improve error message and docs.
> On 12 Mar 2020, at 13:15, Stephan Ewen <se...@apache.org> wrote:
>
> @Andrey about the increase in metaspace size
> - I have no concerns for 1.11.0.
> - For 1.10.1 I am not completely sure, because users expect to upgrade
> that without config adjustments. That might not be possible with that
> change.
>
> On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <az...@gmail.com>
> wrote:
>
>>
>>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
>> job”
>>
>> My understanding that the issue is basically covered by:
>>
>> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error
>> in TaskManager
>> no full consensus there but improving error message for existing task
>> thread fatal handling could be done at least
>>
>> - [FLINK-16406] Increase default value for JVM Metaspace to minimise its
>> OutOfMemoryError
>> see further
>>
>> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically loaded
>> AWS connectors
>> not sure whether this is a blocker but looks close to be resolved
>>
>>> About "FLINK-16406 Increase default value for JVM Metaspace"
>>> - Have we consensus that this is okay for a bugfix release? It changes
>>> setups, takes away memory from heap / managed memory on existing setups
>>> that keep their flink-conf.yaml.
>>
>> My understanding was that increasing to 256m resolved the reported problems
>> and we decided to make the change so I have merged it today as there were
>> no more concerns.
>> If there are concerns I can revert it.
>>
>> On the other hand, I think improving the message error with reference to
>> the metaspace option should help the most
>> because user would not have to read all docs to fix it
>> then maybe this change is not even needed.
>>
>> Best,
>> Andrey
>>
>>> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
>>>
>>> Good idea to go ahead with 1.10.1
>>>
>>> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
>> job"
>>> - I don't think we have consensus on the exact solution, yet, and some
>> of
>>> the changes might also have side effects that are hard to predict, so I
>> am
>>> not sure we should rush this in.
>>>
>>> About "FLINK-16406 Increase default value for JVM Metaspace"
>>> - Have we consensus that this is okay for a bugfix release? It changes
>>> setups, takes away memory from heap / managed memory on existing setups
>>> that keep their flink-conf.yaml.
>>>
>>> We may need to unblock the release form these two issues and think about
>>> having 1.10.2 in the near future.
>>>
>>> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
>>>
>>>> Thanks for the reminder Jark. Will keep an eye on these two.
>>>>
>>>> Best Regards,
>>>> Yu
>>>>
>>>>
>>>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Thanks for driving this release, Yu!
>>>>> +1 to start 1.10.1 release cycle.
>>>>>
>>>>> From the Table SQL module, I think we should also try to get in the
>>>>> following issues:
>>>>> - FLINK-16441: Allow users to override flink-conf parameters from SQL
>> CLI
>>>>> environment
>>>>> this allows users to set e.g. statebackend, watermark interval,
>>>>> exactly-once/at-least-once, in the SQL CLI
>>>>> - FLINK-16217: SQL Client crashed when any uncatched exception is
>> thrown
>>>>> this will improve much experience when using SQL CLI
>>>>>
>>>>> Best,
>>>>> Jark
>>>>>
>>>>>
>>>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
>>>>>
>>>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
>>>> fix
>>>>>> versions and promoted its priority to Critical. Will also watch the
>>>>>> progress of FLINK-16108/FLINK-16408.
>>>>>>
>>>>>> Best Regards,
>>>>>> Yu
>>>>>>
>>>>>>
>>>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Yu,
>>>>>>>
>>>>>>> Thanks for kicking off the 1.10.1 release discussion!
>>>>>>>
>>>>>>> Apart from
>>>>>>> - FLINK-16406 Increase default value for JVM Metaspace to minimise
>>>> its
>>>>>>> OutOfMemoryError
>>>>>>> which should be merged soon
>>>>>>>
>>>>>>> I think we should also try to get in the following issues:
>>>>>>>
>>>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
>>>>> Error
>>>>>> in
>>>>>>> TaskManager
>>>>>>> This should solve the Metaspace problem even in a better way because
>>>>> OOM
>>>>>>> failure should point users to the docs immediately
>>>>>>>
>>>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a slot
>>>>>>> This should give a better protection against class loading leaks
>>>>>>>
>>>>>>> - [FLINK-16018] Improve error reporting when submitting batch job
>>>>>> (instead
>>>>>>> of AskTimeoutException)
>>>>>>> This problem has recently happened for multiple users
>>>>>>>
>>>>>>> Best,
>>>>>>> Andrey
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
>>>>>>>>
>>>>>>>> Some issues are very important, Users are looking forward to them.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Jingsong Lee
>>>>>>>>
>>>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks for driving this release, Yu!
>>>>>>>>>
>>>>>>>>> +1 for starting the 1.10.1 release cycle.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Yangze Guo
>>>>>>>>>
>>>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
>>>>> tonysong820@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Yu,
>>>>>>>>>> Thanks for the explanation.
>>>>>>>>>> I've no concerns. I was just trying to get some inputs for
>>>>>>> prioritizing
>>>>>>>>>> tasks on my side, and ~1month sounds good to me.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you~
>>>>>>>>>>
>>>>>>>>>> Xintong Song
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> bq. what is the time plan for 1.10.1?
>>>>>>>>>>>
>>>>>>>>>>> According to the history, the first patch release of a major
>>>>>>> version
>>>>>>>>> will
>>>>>>>>>>> take ~1month from discussion started, depending on the speed
>>>> of
>>>>>>>> blocker
>>>>>>>>>>> issue resolving:
>>>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on Jul
>>>>> 3rd
>>>>>>> [2]
>>>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on Oct
>>>>> 19th
>>>>>>> [4]
>>>>>>>>>>>
>>>>>>>>>>> We won't rush to match the history of course, but could use
>>>> it
>>>>>> as a
>>>>>>>>>>> reference. And please feel free to let me know if any
>>>> concerns
>>>>>>>> Xintong.
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Yu
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>>>>>>>>>>> [2]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>>>>>>>>>>> [3]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>>>>>>>>>>> [4]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
>>>>>> tonysong820@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
>>>>> release
>>>>>>>>> manager.
>>>>>>>>>>>>
>>>>>>>>>>>> +1 for the proposal.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you~
>>>>>>>>>>>>
>>>>>>>>>>>> Xintong Song
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>>>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for driving this release, Yu!
>>>>>>>>>>>>> +1 on my side
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> Zhijiang
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>> ------------------------------------------------------------------
>>>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>>>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>>>>>>>>>>>>> To:dev <de...@flink.apache.org>
>>>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
>>>> eye
>>>>> on
>>>>>>>> these
>>>>>>>>>>>>> existing blocker issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
>>>>> hequn@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Yu,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
>>>> as
>>>>>> the
>>>>>>>>> release
>>>>>>>>>>>>>> manager!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I found there are some other issues[1] which are marked
>>>>> as
>>>>>> a
>>>>>>>>> blocker:
>>>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
>>>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>>>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
>>>>>> directory
>>>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>>>>> ClassNotFoundException
>>>>>>> when
>>>>>>>>> use
>>>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>>>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
>>>>> batch
>>>>>>> job
>>>>>>>>>>>> (instead
>>>>>>>>>>>>> of
>>>>>>>>>>>>>> AskTimeoutException)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Hequn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>>>> carp84@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Jincheng,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
>>>>>>>>>>> sunjincheng121@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
>>>>> to
>>>>>>> give
>>>>>>>>> you a
>>>>>>>>>>>>> hand
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
>>>> need)
>>>>>> :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>> Jincheng
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It has been almost one month since we released
>>>>> Flink
>>>>>>>>> 1.10.0. We
>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
>>>> the
>>>>>>>>>>> release-1.10
>>>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Most noticeable fixes are:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
>>>> notice
>>>>>>> file
>>>>>>>> in
>>>>>>>>>>>>>>> flink-ml-lib
>>>>>>>>>>>>>>>>> module
>>>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
>>>>>>>>>>>>> flink-state-processor-api
>>>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
>>>>>>>>> HiveCatalog
>>>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
>>>>>>>> TypeExtractor
>>>>>>>>>>> when
>>>>>>>>>>>>>>> mapping
>>>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
>>>> running
>>>>>> in
>>>>>>>>> blink
>>>>>>>>>>>>> planner
>>>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
>>>> reset
>>>>>> on
>>>>>>>> task
>>>>>>>>>>>>> recovery
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> DefaultScheduler
>>>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
>>>>>>> casuing
>>>>>>>>>>>>>>>>> ValidationException: SQL validation failed
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Furthermore, I think the following issues should
>>>> be
>>>>>>>> merged
>>>>>>>>>>> before
>>>>>>>>>>>>>>> 1.10.1
>>>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
>>>>> error
>>>>>> on
>>>>>>>>>>> repeated
>>>>>>>>>>>>> job
>>>>>>>>>>>>>>>>> submission
>>>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
>>>>>> Metaspace
>>>>>>> to
>>>>>>>>>>>> minimise
>>>>>>>>>>>>>> its
>>>>>>>>>>>>>>>>> OutOfMemoryError
>>>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
>>>>> aggregate
>>>>>>>>> results
>>>>>>>>>>> with
>>>>>>>>>>>>>> state
>>>>>>>>>>>>>>>>> clean up
>>>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
>>>> correct
>>>>>>>> unique
>>>>>>>>> key
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> UpsertStreamTableSink
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
>>>>> off
>>>>>>> the
>>>>>>>>>>> release
>>>>>>>>>>>>>>> process
>>>>>>>>>>>>>>>>> once blocker issues are merged. What do you
>>>> think?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If there are any concerns or missing blocker
>>>> issues
>>>>>>> need
>>>>>>>>> to be
>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best, Jingsong Lee
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Stephan Ewen <se...@apache.org>.
@Andrey about the increase in metaspace size
- I have no concerns for 1.11.0.
- For 1.10.1 I am not completely sure, because users expect to upgrade
that without config adjustments. That might not be possible with that
change.
On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin <az...@gmail.com>
wrote:
>
> > About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> job”
>
> My understanding that the issue is basically covered by:
>
> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error
> in TaskManager
> no full consensus there but improving error message for existing task
> thread fatal handling could be done at least
>
> - [FLINK-16406] Increase default value for JVM Metaspace to minimise its
> OutOfMemoryError
> see further
>
> - [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically loaded
> AWS connectors
> not sure whether this is a blocker but looks close to be resolved
>
> > About "FLINK-16406 Increase default value for JVM Metaspace"
> > - Have we consensus that this is okay for a bugfix release? It changes
> > setups, takes away memory from heap / managed memory on existing setups
> > that keep their flink-conf.yaml.
>
> My understanding was that increasing to 256m resolved the reported problems
> and we decided to make the change so I have merged it today as there were
> no more concerns.
> If there are concerns I can revert it.
>
> On the other hand, I think improving the message error with reference to
> the metaspace option should help the most
> because user would not have to read all docs to fix it
> then maybe this change is not even needed.
>
> Best,
> Andrey
>
> > On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
> >
> > Good idea to go ahead with 1.10.1
> >
> > About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> job"
> > - I don't think we have consensus on the exact solution, yet, and some
> of
> > the changes might also have side effects that are hard to predict, so I
> am
> > not sure we should rush this in.
> >
> > About "FLINK-16406 Increase default value for JVM Metaspace"
> > - Have we consensus that this is okay for a bugfix release? It changes
> > setups, takes away memory from heap / managed memory on existing setups
> > that keep their flink-conf.yaml.
> >
> > We may need to unblock the release form these two issues and think about
> > having 1.10.2 in the near future.
> >
> > On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> >
> >> Thanks for the reminder Jark. Will keep an eye on these two.
> >>
> >> Best Regards,
> >> Yu
> >>
> >>
> >> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> >>
> >>> Thanks for driving this release, Yu!
> >>> +1 to start 1.10.1 release cycle.
> >>>
> >>> From the Table SQL module, I think we should also try to get in the
> >>> following issues:
> >>> - FLINK-16441: Allow users to override flink-conf parameters from SQL
> CLI
> >>> environment
> >>> this allows users to set e.g. statebackend, watermark interval,
> >>> exactly-once/at-least-once, in the SQL CLI
> >>> - FLINK-16217: SQL Client crashed when any uncatched exception is
> thrown
> >>> this will improve much experience when using SQL CLI
> >>>
> >>> Best,
> >>> Jark
> >>>
> >>>
> >>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> >>>
> >>>> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
> >> fix
> >>>> versions and promoted its priority to Critical. Will also watch the
> >>>> progress of FLINK-16108/FLINK-16408.
> >>>>
> >>>> Best Regards,
> >>>> Yu
> >>>>
> >>>>
> >>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hi Yu,
> >>>>>
> >>>>> Thanks for kicking off the 1.10.1 release discussion!
> >>>>>
> >>>>> Apart from
> >>>>> - FLINK-16406 Increase default value for JVM Metaspace to minimise
> >> its
> >>>>> OutOfMemoryError
> >>>>> which should be merged soon
> >>>>>
> >>>>> I think we should also try to get in the following issues:
> >>>>>
> >>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> >>> Error
> >>>> in
> >>>>> TaskManager
> >>>>> This should solve the Metaspace problem even in a better way because
> >>> OOM
> >>>>> failure should point users to the docs immediately
> >>>>>
> >>>>> - [FLINK-16408] Bind user code class loader to lifetime of a slot
> >>>>> This should give a better protection against class loading leaks
> >>>>>
> >>>>> - [FLINK-16018] Improve error reporting when submitting batch job
> >>>> (instead
> >>>>> of AskTimeoutException)
> >>>>> This problem has recently happened for multiple users
> >>>>>
> >>>>> Best,
> >>>>> Andrey
> >>>>>
> >>>>>
> >>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> >>>>>>
> >>>>>> Some issues are very important, Users are looking forward to them.
> >>>>>>
> >>>>>> Best,
> >>>>>> Jingsong Lee
> >>>>>>
> >>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> Thanks for driving this release, Yu!
> >>>>>>>
> >>>>>>> +1 for starting the 1.10.1 release cycle.
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Yangze Guo
> >>>>>>>
> >>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> >>> tonysong820@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Yu,
> >>>>>>>> Thanks for the explanation.
> >>>>>>>> I've no concerns. I was just trying to get some inputs for
> >>>>> prioritizing
> >>>>>>>> tasks on my side, and ~1month sounds good to me.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Thank you~
> >>>>>>>>
> >>>>>>>> Xintong Song
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> >> wrote:
> >>>>>>>>
> >>>>>>>>> bq. what is the time plan for 1.10.1?
> >>>>>>>>>
> >>>>>>>>> According to the history, the first patch release of a major
> >>>>> version
> >>>>>>> will
> >>>>>>>>> take ~1month from discussion started, depending on the speed
> >> of
> >>>>>> blocker
> >>>>>>>>> issue resolving:
> >>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on Jul
> >>> 3rd
> >>>>> [2]
> >>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on Oct
> >>> 19th
> >>>>> [4]
> >>>>>>>>>
> >>>>>>>>> We won't rush to match the history of course, but could use
> >> it
> >>>> as a
> >>>>>>>>> reference. And please feel free to let me know if any
> >> concerns
> >>>>>> Xintong.
> >>>>>>>>> Thanks.
> >>>>>>>>>
> >>>>>>>>> Best Regards,
> >>>>>>>>> Yu
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> >>>>>>>>> [2]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> >>>>>>>>> [3]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> >>>>>>>>> [4]
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> >>>> tonysong820@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
> >>> release
> >>>>>>> manager.
> >>>>>>>>>>
> >>>>>>>>>> +1 for the proposal.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> One quick question, what is the time plan for 1.10.1?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thank you~
> >>>>>>>>>>
> >>>>>>>>>> Xintong Song
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> >>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thanks for driving this release, Yu!
> >>>>>>>>>>> +1 on my side
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Best,
> >>>>>>>>>>> Zhijiang
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>> ------------------------------------------------------------------
> >>>>>>>>>>> From:Yu Li <ca...@gmail.com>
> >>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
> >>>>>>>>>>> To:dev <de...@flink.apache.org>
> >>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
> >> eye
> >>> on
> >>>>>> these
> >>>>>>>>>>> existing blocker issues.
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Yu
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> >>> hequn@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Yu,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
> >> as
> >>>> the
> >>>>>>> release
> >>>>>>>>>>>> manager!
> >>>>>>>>>>>>
> >>>>>>>>>>>> I found there are some other issues[1] which are marked
> >>> as
> >>>> a
> >>>>>>> blocker:
> >>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
> >>>>>>>>>>>> - FLINK-16262 Class loader problem with
> >>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> >>>> directory
> >>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
> >>> ClassNotFoundException
> >>>>> when
> >>>>>>> use
> >>>>>>>>>>>> flink-sql-connector-elasticsearch7
> >>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
> >>> batch
> >>>>> job
> >>>>>>>>>> (instead
> >>>>>>>>>>> of
> >>>>>>>>>>>> AskTimeoutException)
> >>>>>>>>>>>>
> >>>>>>>>>>>> These may also need to be resolved in 1.10.1.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best,
> >>>>>>>>>>>> Hequn
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1]
> >>>>>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> >> carp84@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Jincheng,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> >>>>>>>>> sunjincheng121@gmail.com
> >>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
> >>> to
> >>>>> give
> >>>>>>> you a
> >>>>>>>>>>> hand
> >>>>>>>>>>>> at
> >>>>>>>>>>>>>> the last stage when the RC is finished.(If you
> >> need)
> >>>> :)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>> Jincheng
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi All,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It has been almost one month since we released
> >>> Flink
> >>>>>>> 1.10.0. We
> >>>>>>>>>>>> already
> >>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
> >> the
> >>>>>>>>> release-1.10
> >>>>>>>>>>>>> branch,
> >>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Most noticeable fixes are:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
> >> notice
> >>>>> file
> >>>>>> in
> >>>>>>>>>>>>> flink-ml-lib
> >>>>>>>>>>>>>>> module
> >>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
> >>>>>>>>>>> flink-state-processor-api
> >>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
> >>>>>>> HiveCatalog
> >>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
> >>>>>> TypeExtractor
> >>>>>>>>> when
> >>>>>>>>>>>>> mapping
> >>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
> >> running
> >>>> in
> >>>>>>> blink
> >>>>>>>>>>> planner
> >>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
> >> reset
> >>>> on
> >>>>>> task
> >>>>>>>>>>> recovery
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>> DefaultScheduler
> >>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
> >>>>> casuing
> >>>>>>>>>>>>>>> ValidationException: SQL validation failed
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Furthermore, I think the following issues should
> >> be
> >>>>>> merged
> >>>>>>>>> before
> >>>>>>>>>>>>> 1.10.1
> >>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
> >>> error
> >>>> on
> >>>>>>>>> repeated
> >>>>>>>>>>> job
> >>>>>>>>>>>>>>> submission
> >>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
> >>>> Metaspace
> >>>>> to
> >>>>>>>>>> minimise
> >>>>>>>>>>>> its
> >>>>>>>>>>>>>>> OutOfMemoryError
> >>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
> >>> aggregate
> >>>>>>> results
> >>>>>>>>> with
> >>>>>>>>>>>> state
> >>>>>>>>>>>>>>> clean up
> >>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
> >> correct
> >>>>>> unique
> >>>>>>> key
> >>>>>>>>>> for
> >>>>>>>>>>>>>>> UpsertStreamTableSink
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I would volunteer as the release manager and kick
> >>> off
> >>>>> the
> >>>>>>>>> release
> >>>>>>>>>>>>> process
> >>>>>>>>>>>>>>> once blocker issues are merged. What do you
> >> think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If there are any concerns or missing blocker
> >> issues
> >>>>> need
> >>>>>>> to be
> >>>>>>>>>>> fixed
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>> Yu
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best, Jingsong Lee
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Andrey Zagrebin <az...@gmail.com>.
> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job”
My understanding that the issue is basically covered by:
- [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error in TaskManager
no full consensus there but improving error message for existing task thread fatal handling could be done at least
- [FLINK-16406] Increase default value for JVM Metaspace to minimise its OutOfMemoryError
see further
- [FLINK-16246] Exclude "SdkMBeanRegistrySupport" from dynamically loaded AWS connectors
not sure whether this is a blocker but looks close to be resolved
> About "FLINK-16406 Increase default value for JVM Metaspace"
> - Have we consensus that this is okay for a bugfix release? It changes
> setups, takes away memory from heap / managed memory on existing setups
> that keep their flink-conf.yaml.
My understanding was that increasing to 256m resolved the reported problems
and we decided to make the change so I have merged it today as there were no more concerns.
If there are concerns I can revert it.
On the other hand, I think improving the message error with reference to the metaspace option should help the most
because user would not have to read all docs to fix it
then maybe this change is not even needed.
Best,
Andrey
> On 12 Mar 2020, at 12:28, Stephan Ewen <se...@apache.org> wrote:
>
> Good idea to go ahead with 1.10.1
>
> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job"
> - I don't think we have consensus on the exact solution, yet, and some of
> the changes might also have side effects that are hard to predict, so I am
> not sure we should rush this in.
>
> About "FLINK-16406 Increase default value for JVM Metaspace"
> - Have we consensus that this is okay for a bugfix release? It changes
> setups, takes away memory from heap / managed memory on existing setups
> that keep their flink-conf.yaml.
>
> We may need to unblock the release form these two issues and think about
> having 1.10.2 in the near future.
>
> On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
>
>> Thanks for the reminder Jark. Will keep an eye on these two.
>>
>> Best Regards,
>> Yu
>>
>>
>> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
>>
>>> Thanks for driving this release, Yu!
>>> +1 to start 1.10.1 release cycle.
>>>
>>> From the Table SQL module, I think we should also try to get in the
>>> following issues:
>>> - FLINK-16441: Allow users to override flink-conf parameters from SQL CLI
>>> environment
>>> this allows users to set e.g. statebackend, watermark interval,
>>> exactly-once/at-least-once, in the SQL CLI
>>> - FLINK-16217: SQL Client crashed when any uncatched exception is thrown
>>> this will improve much experience when using SQL CLI
>>>
>>> Best,
>>> Jark
>>>
>>>
>>> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
>>>
>>>> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
>> fix
>>>> versions and promoted its priority to Critical. Will also watch the
>>>> progress of FLINK-16108/FLINK-16408.
>>>>
>>>> Best Regards,
>>>> Yu
>>>>
>>>>
>>>> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Yu,
>>>>>
>>>>> Thanks for kicking off the 1.10.1 release discussion!
>>>>>
>>>>> Apart from
>>>>> - FLINK-16406 Increase default value for JVM Metaspace to minimise
>> its
>>>>> OutOfMemoryError
>>>>> which should be merged soon
>>>>>
>>>>> I think we should also try to get in the following issues:
>>>>>
>>>>> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
>>> Error
>>>> in
>>>>> TaskManager
>>>>> This should solve the Metaspace problem even in a better way because
>>> OOM
>>>>> failure should point users to the docs immediately
>>>>>
>>>>> - [FLINK-16408] Bind user code class loader to lifetime of a slot
>>>>> This should give a better protection against class loading leaks
>>>>>
>>>>> - [FLINK-16018] Improve error reporting when submitting batch job
>>>> (instead
>>>>> of AskTimeoutException)
>>>>> This problem has recently happened for multiple users
>>>>>
>>>>> Best,
>>>>> Andrey
>>>>>
>>>>>
>>>>> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
>>>>>>
>>>>>> Some issues are very important, Users are looking forward to them.
>>>>>>
>>>>>> Best,
>>>>>> Jingsong Lee
>>>>>>
>>>>>> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
>>> wrote:
>>>>>>
>>>>>>> Thanks for driving this release, Yu!
>>>>>>>
>>>>>>> +1 for starting the 1.10.1 release cycle.
>>>>>>>
>>>>>>> Best,
>>>>>>> Yangze Guo
>>>>>>>
>>>>>>> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
>>> tonysong820@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Yu,
>>>>>>>> Thanks for the explanation.
>>>>>>>> I've no concerns. I was just trying to get some inputs for
>>>>> prioritizing
>>>>>>>> tasks on my side, and ~1month sounds good to me.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you~
>>>>>>>>
>>>>>>>> Xintong Song
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
>> wrote:
>>>>>>>>
>>>>>>>>> bq. what is the time plan for 1.10.1?
>>>>>>>>>
>>>>>>>>> According to the history, the first patch release of a major
>>>>> version
>>>>>>> will
>>>>>>>>> take ~1month from discussion started, depending on the speed
>> of
>>>>>> blocker
>>>>>>>>> issue resolving:
>>>>>>>>> * 1.8.1: started discussion on May 28th [1], released on Jul
>>> 3rd
>>>>> [2]
>>>>>>>>> * 1.9.1: started discussion on Sep 23rd [3], released on Oct
>>> 19th
>>>>> [4]
>>>>>>>>>
>>>>>>>>> We won't rush to match the history of course, but could use
>> it
>>>> as a
>>>>>>>>> reference. And please feel free to let me know if any
>> concerns
>>>>>> Xintong.
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Yu
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
>>>>>>>>> [2]
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
>>>>>>>>> [3]
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
>>>>>>>>> [4]
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, 11 Mar 2020 at 11:54, Xintong Song <
>>>> tonysong820@gmail.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Yu, for the kick off and volunteering to be the
>>> release
>>>>>>> manager.
>>>>>>>>>>
>>>>>>>>>> +1 for the proposal.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> One quick question, what is the time plan for 1.10.1?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you~
>>>>>>>>>>
>>>>>>>>>> Xintong Song
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
>>>>>>>>>> <wa...@aliyun.com.invalid> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks for driving this release, Yu!
>>>>>>>>>>> +1 on my side
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Zhijiang
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> ------------------------------------------------------------------
>>>>>>>>>>> From:Yu Li <ca...@gmail.com>
>>>>>>>>>>> Send Time:2020 Mar. 10 (Tue.) 20:25
>>>>>>>>>>> To:dev <de...@flink.apache.org>
>>>>>>>>>>> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the supplement Hequn. Yes will also keep an
>> eye
>>> on
>>>>>> these
>>>>>>>>>>> existing blocker issues.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Yu
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
>>> hequn@apache.org>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Yu,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks a lot for raising the discussion and volunteer
>> as
>>>> the
>>>>>>> release
>>>>>>>>>>>> manager!
>>>>>>>>>>>>
>>>>>>>>>>>> I found there are some other issues[1] which are marked
>>> as
>>>> a
>>>>>>> blocker:
>>>>>>>>>>>> - FLINK-16454 Update the copyright year in NOTICE files
>>>>>>>>>>>> - FLINK-16262 Class loader problem with
>>>>>>>>>>>> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
>>>> directory
>>>>>>>>>>>> - FLINK-16170 SearchTemplateRequest
>>> ClassNotFoundException
>>>>> when
>>>>>>> use
>>>>>>>>>>>> flink-sql-connector-elasticsearch7
>>>>>>>>>>>> - FLINK-16018 Improve error reporting when submitting
>>> batch
>>>>> job
>>>>>>>>>> (instead
>>>>>>>>>>> of
>>>>>>>>>>>> AskTimeoutException)
>>>>>>>>>>>>
>>>>>>>>>>>> These may also need to be resolved in 1.10.1.
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Hequn
>>>>>>>>>>>>
>>>>>>>>>>>> [1]
>>>>>>> https://issues.apache.org/jira/projects/FLINK/versions/12346891
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
>> carp84@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Jincheng,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, your help would be very helpful. Thanks a lot!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, 10 Mar 2020 at 18:24, jincheng sun <
>>>>>>>>> sunjincheng121@gmail.com
>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for bring up the discussion Yu. I would like
>>> to
>>>>> give
>>>>>>> you a
>>>>>>>>>>> hand
>>>>>>>>>>>> at
>>>>>>>>>>>>>> the last stage when the RC is finished.(If you
>> need)
>>>> :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>> Jincheng
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It has been almost one month since we released
>>> Flink
>>>>>>> 1.10.0. We
>>>>>>>>>>>> already
>>>>>>>>>>>>>>> have more than 40 resolved improvements/bugs in
>> the
>>>>>>>>> release-1.10
>>>>>>>>>>>>> branch,
>>>>>>>>>>>>>>> and I propose to start the 1.10.1 release cycle.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Most noticeable fixes are:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - FLINK-16241 [legal] Remove the license and
>> notice
>>>>> file
>>>>>> in
>>>>>>>>>>>>> flink-ml-lib
>>>>>>>>>>>>>>> module
>>>>>>>>>>>>>>> - FLINK-16313 Fix RocksDB resource leak in
>>>>>>>>>>> flink-state-processor-api
>>>>>>>>>>>>>>> - FLINK-16161 Statistics zero should be known in
>>>>>>> HiveCatalog
>>>>>>>>>>>>>>> - FLINK-2336 ArrayIndexOufOBoundsException in
>>>>>> TypeExtractor
>>>>>>>>> when
>>>>>>>>>>>>> mapping
>>>>>>>>>>>>>>> - FLINK-16108 StreamSQLExample is failed if
>> running
>>>> in
>>>>>>> blink
>>>>>>>>>>> planner
>>>>>>>>>>>>>>> - FLINK-16139 Co-location constraints are not
>> reset
>>>> on
>>>>>> task
>>>>>>>>>>> recovery
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> DefaultScheduler
>>>>>>>>>>>>>>> - FLINK-16414 Create udaf/udtf function using sql
>>>>> casuing
>>>>>>>>>>>>>>> ValidationException: SQL validation failed
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Furthermore, I think the following issues should
>> be
>>>>>> merged
>>>>>>>>> before
>>>>>>>>>>>>> 1.10.1
>>>>>>>>>>>>>>> release (especially the Metaspace OOM issue):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - FLINK-16142 Memory Leak causes Metaspace OOM
>>> error
>>>> on
>>>>>>>>> repeated
>>>>>>>>>>> job
>>>>>>>>>>>>>>> submission
>>>>>>>>>>>>>>> - FLINK-16406 Increase default value for JVM
>>>> Metaspace
>>>>> to
>>>>>>>>>> minimise
>>>>>>>>>>>> its
>>>>>>>>>>>>>>> OutOfMemoryError
>>>>>>>>>>>>>>> - FLINK-16047 Blink planner produces wrong
>>> aggregate
>>>>>>> results
>>>>>>>>> with
>>>>>>>>>>>> state
>>>>>>>>>>>>>>> clean up
>>>>>>>>>>>>>>> - FLINK-16070 Blink planner can not extract
>> correct
>>>>>> unique
>>>>>>> key
>>>>>>>>>> for
>>>>>>>>>>>>>>> UpsertStreamTableSink
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would volunteer as the release manager and kick
>>> off
>>>>> the
>>>>>>>>> release
>>>>>>>>>>>>> process
>>>>>>>>>>>>>>> once blocker issues are merged. What do you
>> think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If there are any concerns or missing blocker
>> issues
>>>>> need
>>>>>>> to be
>>>>>>>>>>> fixed
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> 1.10.1, please let me know. Thanks.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>> Yu
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best, Jingsong Lee
>>>>>>
>>>>>
>>>>
>>>
>>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Stephan Ewen <se...@apache.org>.
Good idea to go ahead with 1.10.1
About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job"
- I don't think we have consensus on the exact solution, yet, and some of
the changes might also have side effects that are hard to predict, so I am
not sure we should rush this in.
About "FLINK-16406 Increase default value for JVM Metaspace"
- Have we consensus that this is okay for a bugfix release? It changes
setups, takes away memory from heap / managed memory on existing setups
that keep their flink-conf.yaml.
We may need to unblock the release form these two issues and think about
having 1.10.2 in the near future.
On Thu, Mar 12, 2020 at 7:15 AM Yu Li <ca...@gmail.com> wrote:
> Thanks for the reminder Jark. Will keep an eye on these two.
>
> Best Regards,
> Yu
>
>
> On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
>
> > Thanks for driving this release, Yu!
> > +1 to start 1.10.1 release cycle.
> >
> > From the Table SQL module, I think we should also try to get in the
> > following issues:
> > - FLINK-16441: Allow users to override flink-conf parameters from SQL CLI
> > environment
> > this allows users to set e.g. statebackend, watermark interval,
> > exactly-once/at-least-once, in the SQL CLI
> > - FLINK-16217: SQL Client crashed when any uncatched exception is thrown
> > this will improve much experience when using SQL CLI
> >
> > Best,
> > Jark
> >
> >
> > On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> >
> > > Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225
> fix
> > > versions and promoted its priority to Critical. Will also watch the
> > > progress of FLINK-16108/FLINK-16408.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
> > > wrote:
> > >
> > > > Hi Yu,
> > > >
> > > > Thanks for kicking off the 1.10.1 release discussion!
> > > >
> > > > Apart from
> > > > - FLINK-16406 Increase default value for JVM Metaspace to minimise
> its
> > > > OutOfMemoryError
> > > > which should be merged soon
> > > >
> > > > I think we should also try to get in the following issues:
> > > >
> > > > - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> > Error
> > > in
> > > > TaskManager
> > > > This should solve the Metaspace problem even in a better way because
> > OOM
> > > > failure should point users to the docs immediately
> > > >
> > > > - [FLINK-16408] Bind user code class loader to lifetime of a slot
> > > > This should give a better protection against class loading leaks
> > > >
> > > > - [FLINK-16018] Improve error reporting when submitting batch job
> > > (instead
> > > > of AskTimeoutException)
> > > > This problem has recently happened for multiple users
> > > >
> > > > Best,
> > > > Andrey
> > > >
> > > >
> > > > On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > > > >
> > > > > Some issues are very important, Users are looking forward to them.
> > > > >
> > > > > Best,
> > > > > Jingsong Lee
> > > > >
> > > > > On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
> > wrote:
> > > > >
> > > > > > Thanks for driving this release, Yu!
> > > > > >
> > > > > > +1 for starting the 1.10.1 release cycle.
> > > > > >
> > > > > > Best,
> > > > > > Yangze Guo
> > > > > >
> > > > > > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> > tonysong820@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Yu,
> > > > > > > Thanks for the explanation.
> > > > > > > I've no concerns. I was just trying to get some inputs for
> > > > prioritizing
> > > > > > > tasks on my side, and ~1month sounds good to me.
> > > > > > >
> > > > > > >
> > > > > > > Thank you~
> > > > > > >
> > > > > > > Xintong Song
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com>
> wrote:
> > > > > > >
> > > > > > > > bq. what is the time plan for 1.10.1?
> > > > > > > >
> > > > > > > > According to the history, the first patch release of a major
> > > > version
> > > > > > will
> > > > > > > > take ~1month from discussion started, depending on the speed
> of
> > > > > blocker
> > > > > > > > issue resolving:
> > > > > > > > * 1.8.1: started discussion on May 28th [1], released on Jul
> > 3rd
> > > > [2]
> > > > > > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct
> > 19th
> > > > [4]
> > > > > > > >
> > > > > > > > We won't rush to match the history of course, but could use
> it
> > > as a
> > > > > > > > reference. And please feel free to let me know if any
> concerns
> > > > > Xintong.
> > > > > > > > Thanks.
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Yu
> > > > > > > >
> > > > > > > > [1]
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > > > [2]
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > > > [3]
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > > > [4]
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > > tonysong820@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks Yu, for the kick off and volunteering to be the
> > release
> > > > > > manager.
> > > > > > > > >
> > > > > > > > > +1 for the proposal.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > One quick question, what is the time plan for 1.10.1?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thank you~
> > > > > > > > >
> > > > > > > > > Xintong Song
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > > > > <wa...@aliyun.com.invalid> wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for driving this release, Yu!
> > > > > > > > > > +1 on my side
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Zhijiang
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > ------------------------------------------------------------------
> > > > > > > > > > From:Yu Li <ca...@gmail.com>
> > > > > > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > > > > To:dev <de...@flink.apache.org>
> > > > > > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > > > > >
> > > > > > > > > > Thanks for the supplement Hequn. Yes will also keep an
> eye
> > on
> > > > > these
> > > > > > > > > > existing blocker issues.
> > > > > > > > > >
> > > > > > > > > > Best Regards,
> > > > > > > > > > Yu
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> > hequn@apache.org>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Yu,
> > > > > > > > > > >
> > > > > > > > > > > Thanks a lot for raising the discussion and volunteer
> as
> > > the
> > > > > > release
> > > > > > > > > > > manager!
> > > > > > > > > > >
> > > > > > > > > > > I found there are some other issues[1] which are marked
> > as
> > > a
> > > > > > blocker:
> > > > > > > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > > > > > > - FLINK-16262 Class loader problem with
> > > > > > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > > directory
> > > > > > > > > > > - FLINK-16170 SearchTemplateRequest
> > ClassNotFoundException
> > > > when
> > > > > > use
> > > > > > > > > > > flink-sql-connector-elasticsearch7
> > > > > > > > > > > - FLINK-16018 Improve error reporting when submitting
> > batch
> > > > job
> > > > > > > > > (instead
> > > > > > > > > > of
> > > > > > > > > > > AskTimeoutException)
> > > > > > > > > > >
> > > > > > > > > > > These may also need to be resolved in 1.10.1.
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Hequn
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <
> carp84@gmail.com>
> > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Jincheng,
> > > > > > > > > > > >
> > > > > > > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > > > > > > >
> > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > Yu
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > > > sunjincheng121@gmail.com
> > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for bring up the discussion Yu. I would like
> > to
> > > > give
> > > > > > you a
> > > > > > > > > > hand
> > > > > > > > > > > at
> > > > > > > > > > > > > the last stage when the RC is finished.(If you
> need)
> > > :)
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Jincheng
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi All,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It has been almost one month since we released
> > Flink
> > > > > > 1.10.0. We
> > > > > > > > > > > already
> > > > > > > > > > > > > > have more than 40 resolved improvements/bugs in
> the
> > > > > > > > release-1.10
> > > > > > > > > > > > branch,
> > > > > > > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Most noticeable fixes are:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - FLINK-16241 [legal] Remove the license and
> notice
> > > > file
> > > > > in
> > > > > > > > > > > > flink-ml-lib
> > > > > > > > > > > > > > module
> > > > > > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > > > > flink-state-processor-api
> > > > > > > > > > > > > > - FLINK-16161 Statistics zero should be known in
> > > > > > HiveCatalog
> > > > > > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > > TypeExtractor
> > > > > > > > when
> > > > > > > > > > > > mapping
> > > > > > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if
> running
> > > in
> > > > > > blink
> > > > > > > > > > planner
> > > > > > > > > > > > > > - FLINK-16139 Co-location constraints are not
> reset
> > > on
> > > > > task
> > > > > > > > > > recovery
> > > > > > > > > > > in
> > > > > > > > > > > > > > DefaultScheduler
> > > > > > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql
> > > > casuing
> > > > > > > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Furthermore, I think the following issues should
> be
> > > > > merged
> > > > > > > > before
> > > > > > > > > > > > 1.10.1
> > > > > > > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM
> > error
> > > on
> > > > > > > > repeated
> > > > > > > > > > job
> > > > > > > > > > > > > > submission
> > > > > > > > > > > > > > - FLINK-16406 Increase default value for JVM
> > > Metaspace
> > > > to
> > > > > > > > > minimise
> > > > > > > > > > > its
> > > > > > > > > > > > > > OutOfMemoryError
> > > > > > > > > > > > > > - FLINK-16047 Blink planner produces wrong
> > aggregate
> > > > > > results
> > > > > > > > with
> > > > > > > > > > > state
> > > > > > > > > > > > > > clean up
> > > > > > > > > > > > > > - FLINK-16070 Blink planner can not extract
> correct
> > > > > unique
> > > > > > key
> > > > > > > > > for
> > > > > > > > > > > > > > UpsertStreamTableSink
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I would volunteer as the release manager and kick
> > off
> > > > the
> > > > > > > > release
> > > > > > > > > > > > process
> > > > > > > > > > > > > > once blocker issues are merged. What do you
> think?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If there are any concerns or missing blocker
> issues
> > > > need
> > > > > > to be
> > > > > > > > > > fixed
> > > > > > > > > > > in
> > > > > > > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > > > Yu
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best, Jingsong Lee
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the reminder Jark. Will keep an eye on these two.
Best Regards,
Yu
On Thu, 12 Mar 2020 at 12:32, Jark Wu <im...@gmail.com> wrote:
> Thanks for driving this release, Yu!
> +1 to start 1.10.1 release cycle.
>
> From the Table SQL module, I think we should also try to get in the
> following issues:
> - FLINK-16441: Allow users to override flink-conf parameters from SQL CLI
> environment
> this allows users to set e.g. statebackend, watermark interval,
> exactly-once/at-least-once, in the SQL CLI
> - FLINK-16217: SQL Client crashed when any uncatched exception is thrown
> this will improve much experience when using SQL CLI
>
> Best,
> Jark
>
>
> On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
>
> > Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225 fix
> > versions and promoted its priority to Critical. Will also watch the
> > progress of FLINK-16108/FLINK-16408.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
> > wrote:
> >
> > > Hi Yu,
> > >
> > > Thanks for kicking off the 1.10.1 release discussion!
> > >
> > > Apart from
> > > - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> > > OutOfMemoryError
> > > which should be merged soon
> > >
> > > I think we should also try to get in the following issues:
> > >
> > > - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal
> Error
> > in
> > > TaskManager
> > > This should solve the Metaspace problem even in a better way because
> OOM
> > > failure should point users to the docs immediately
> > >
> > > - [FLINK-16408] Bind user code class loader to lifetime of a slot
> > > This should give a better protection against class loading leaks
> > >
> > > - [FLINK-16018] Improve error reporting when submitting batch job
> > (instead
> > > of AskTimeoutException)
> > > This problem has recently happened for multiple users
> > >
> > > Best,
> > > Andrey
> > >
> > >
> > > On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
> > > wrote:
> > >
> > > > Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > > >
> > > > Some issues are very important, Users are looking forward to them.
> > > >
> > > > Best,
> > > > Jingsong Lee
> > > >
> > > > On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com>
> wrote:
> > > >
> > > > > Thanks for driving this release, Yu!
> > > > >
> > > > > +1 for starting the 1.10.1 release cycle.
> > > > >
> > > > > Best,
> > > > > Yangze Guo
> > > > >
> > > > > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <
> tonysong820@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Yu,
> > > > > > Thanks for the explanation.
> > > > > > I've no concerns. I was just trying to get some inputs for
> > > prioritizing
> > > > > > tasks on my side, and ~1month sounds good to me.
> > > > > >
> > > > > >
> > > > > > Thank you~
> > > > > >
> > > > > > Xintong Song
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> > > > > >
> > > > > > > bq. what is the time plan for 1.10.1?
> > > > > > >
> > > > > > > According to the history, the first patch release of a major
> > > version
> > > > > will
> > > > > > > take ~1month from discussion started, depending on the speed of
> > > > blocker
> > > > > > > issue resolving:
> > > > > > > * 1.8.1: started discussion on May 28th [1], released on Jul
> 3rd
> > > [2]
> > > > > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct
> 19th
> > > [4]
> > > > > > >
> > > > > > > We won't rush to match the history of course, but could use it
> > as a
> > > > > > > reference. And please feel free to let me know if any concerns
> > > > Xintong.
> > > > > > > Thanks.
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Yu
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > > [3]
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > > [4]
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > > >
> > > > > > >
> > > > > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> > tonysong820@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Thanks Yu, for the kick off and volunteering to be the
> release
> > > > > manager.
> > > > > > > >
> > > > > > > > +1 for the proposal.
> > > > > > > >
> > > > > > > >
> > > > > > > > One quick question, what is the time plan for 1.10.1?
> > > > > > > >
> > > > > > > >
> > > > > > > > Thank you~
> > > > > > > >
> > > > > > > > Xintong Song
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > > > <wa...@aliyun.com.invalid> wrote:
> > > > > > > >
> > > > > > > > > Thanks for driving this release, Yu!
> > > > > > > > > +1 on my side
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Zhijiang
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > ------------------------------------------------------------------
> > > > > > > > > From:Yu Li <ca...@gmail.com>
> > > > > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > > > To:dev <de...@flink.apache.org>
> > > > > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > > > >
> > > > > > > > > Thanks for the supplement Hequn. Yes will also keep an eye
> on
> > > > these
> > > > > > > > > existing blocker issues.
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Yu
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <
> hequn@apache.org>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Yu,
> > > > > > > > > >
> > > > > > > > > > Thanks a lot for raising the discussion and volunteer as
> > the
> > > > > release
> > > > > > > > > > manager!
> > > > > > > > > >
> > > > > > > > > > I found there are some other issues[1] which are marked
> as
> > a
> > > > > blocker:
> > > > > > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > > > > > - FLINK-16262 Class loader problem with
> > > > > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> > directory
> > > > > > > > > > - FLINK-16170 SearchTemplateRequest
> ClassNotFoundException
> > > when
> > > > > use
> > > > > > > > > > flink-sql-connector-elasticsearch7
> > > > > > > > > > - FLINK-16018 Improve error reporting when submitting
> batch
> > > job
> > > > > > > > (instead
> > > > > > > > > of
> > > > > > > > > > AskTimeoutException)
> > > > > > > > > >
> > > > > > > > > > These may also need to be resolved in 1.10.1.
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Hequn
> > > > > > > > > >
> > > > > > > > > > [1]
> > > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com>
> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Jincheng,
> > > > > > > > > > >
> > > > > > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > > > > > >
> > > > > > > > > > > Best Regards,
> > > > > > > > > > > Yu
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > > sunjincheng121@gmail.com
> > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Thanks for bring up the discussion Yu. I would like
> to
> > > give
> > > > > you a
> > > > > > > > > hand
> > > > > > > > > > at
> > > > > > > > > > > > the last stage when the RC is finished.(If you need)
> > :)
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Jincheng
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi All,
> > > > > > > > > > > > >
> > > > > > > > > > > > > It has been almost one month since we released
> Flink
> > > > > 1.10.0. We
> > > > > > > > > > already
> > > > > > > > > > > > > have more than 40 resolved improvements/bugs in the
> > > > > > > release-1.10
> > > > > > > > > > > branch,
> > > > > > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Most noticeable fixes are:
> > > > > > > > > > > > >
> > > > > > > > > > > > > - FLINK-16241 [legal] Remove the license and notice
> > > file
> > > > in
> > > > > > > > > > > flink-ml-lib
> > > > > > > > > > > > > module
> > > > > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > > > flink-state-processor-api
> > > > > > > > > > > > > - FLINK-16161 Statistics zero should be known in
> > > > > HiveCatalog
> > > > > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in
> > > > TypeExtractor
> > > > > > > when
> > > > > > > > > > > mapping
> > > > > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if running
> > in
> > > > > blink
> > > > > > > > > planner
> > > > > > > > > > > > > - FLINK-16139 Co-location constraints are not reset
> > on
> > > > task
> > > > > > > > > recovery
> > > > > > > > > > in
> > > > > > > > > > > > > DefaultScheduler
> > > > > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql
> > > casuing
> > > > > > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > > > > > >
> > > > > > > > > > > > > Furthermore, I think the following issues should be
> > > > merged
> > > > > > > before
> > > > > > > > > > > 1.10.1
> > > > > > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > > > > > >
> > > > > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM
> error
> > on
> > > > > > > repeated
> > > > > > > > > job
> > > > > > > > > > > > > submission
> > > > > > > > > > > > > - FLINK-16406 Increase default value for JVM
> > Metaspace
> > > to
> > > > > > > > minimise
> > > > > > > > > > its
> > > > > > > > > > > > > OutOfMemoryError
> > > > > > > > > > > > > - FLINK-16047 Blink planner produces wrong
> aggregate
> > > > > results
> > > > > > > with
> > > > > > > > > > state
> > > > > > > > > > > > > clean up
> > > > > > > > > > > > > - FLINK-16070 Blink planner can not extract correct
> > > > unique
> > > > > key
> > > > > > > > for
> > > > > > > > > > > > > UpsertStreamTableSink
> > > > > > > > > > > > >
> > > > > > > > > > > > > I would volunteer as the release manager and kick
> off
> > > the
> > > > > > > release
> > > > > > > > > > > process
> > > > > > > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > > > > > > >
> > > > > > > > > > > > > If there are any concerns or missing blocker issues
> > > need
> > > > > to be
> > > > > > > > > fixed
> > > > > > > > > > in
> > > > > > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > > Yu
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best, Jingsong Lee
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Jark Wu <im...@gmail.com>.
Thanks for driving this release, Yu!
+1 to start 1.10.1 release cycle.
From the Table SQL module, I think we should also try to get in the
following issues:
- FLINK-16441: Allow users to override flink-conf parameters from SQL CLI
environment
this allows users to set e.g. statebackend, watermark interval,
exactly-once/at-least-once, in the SQL CLI
- FLINK-16217: SQL Client crashed when any uncatched exception is thrown
this will improve much experience when using SQL CLI
Best,
Jark
On Wed, 11 Mar 2020 at 20:37, Yu Li <ca...@gmail.com> wrote:
> Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225 fix
> versions and promoted its priority to Critical. Will also watch the
> progress of FLINK-16108/FLINK-16408.
>
> Best Regards,
> Yu
>
>
> On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org>
> wrote:
>
> > Hi Yu,
> >
> > Thanks for kicking off the 1.10.1 release discussion!
> >
> > Apart from
> > - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> > OutOfMemoryError
> > which should be merged soon
> >
> > I think we should also try to get in the following issues:
> >
> > - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error
> in
> > TaskManager
> > This should solve the Metaspace problem even in a better way because OOM
> > failure should point users to the docs immediately
> >
> > - [FLINK-16408] Bind user code class loader to lifetime of a slot
> > This should give a better protection against class loading leaks
> >
> > - [FLINK-16018] Improve error reporting when submitting batch job
> (instead
> > of AskTimeoutException)
> > This problem has recently happened for multiple users
> >
> > Best,
> > Andrey
> >
> >
> > On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
> > wrote:
> >
> > > Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> > >
> > > Some issues are very important, Users are looking forward to them.
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com> wrote:
> > >
> > > > Thanks for driving this release, Yu!
> > > >
> > > > +1 for starting the 1.10.1 release cycle.
> > > >
> > > > Best,
> > > > Yangze Guo
> > > >
> > > > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <to...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Yu,
> > > > > Thanks for the explanation.
> > > > > I've no concerns. I was just trying to get some inputs for
> > prioritizing
> > > > > tasks on my side, and ~1month sounds good to me.
> > > > >
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> > > > >
> > > > > > bq. what is the time plan for 1.10.1?
> > > > > >
> > > > > > According to the history, the first patch release of a major
> > version
> > > > will
> > > > > > take ~1month from discussion started, depending on the speed of
> > > blocker
> > > > > > issue resolving:
> > > > > > * 1.8.1: started discussion on May 28th [1], released on Jul 3rd
> > [2]
> > > > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th
> > [4]
> > > > > >
> > > > > > We won't rush to match the history of course, but could use it
> as a
> > > > > > reference. And please feel free to let me know if any concerns
> > > Xintong.
> > > > > > Thanks.
> > > > > >
> > > > > > Best Regards,
> > > > > > Yu
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > > [2]
> > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > > [3]
> > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > > [4]
> > > > > >
> > > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > > >
> > > > > >
> > > > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <
> tonysong820@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Thanks Yu, for the kick off and volunteering to be the release
> > > > manager.
> > > > > > >
> > > > > > > +1 for the proposal.
> > > > > > >
> > > > > > >
> > > > > > > One quick question, what is the time plan for 1.10.1?
> > > > > > >
> > > > > > >
> > > > > > > Thank you~
> > > > > > >
> > > > > > > Xintong Song
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > > <wa...@aliyun.com.invalid> wrote:
> > > > > > >
> > > > > > > > Thanks for driving this release, Yu!
> > > > > > > > +1 on my side
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Zhijiang
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > ------------------------------------------------------------------
> > > > > > > > From:Yu Li <ca...@gmail.com>
> > > > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > > To:dev <de...@flink.apache.org>
> > > > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > > >
> > > > > > > > Thanks for the supplement Hequn. Yes will also keep an eye on
> > > these
> > > > > > > > existing blocker issues.
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Yu
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org>
> > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Yu,
> > > > > > > > >
> > > > > > > > > Thanks a lot for raising the discussion and volunteer as
> the
> > > > release
> > > > > > > > > manager!
> > > > > > > > >
> > > > > > > > > I found there are some other issues[1] which are marked as
> a
> > > > blocker:
> > > > > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > > > > - FLINK-16262 Class loader problem with
> > > > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib
> directory
> > > > > > > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException
> > when
> > > > use
> > > > > > > > > flink-sql-connector-elasticsearch7
> > > > > > > > > - FLINK-16018 Improve error reporting when submitting batch
> > job
> > > > > > > (instead
> > > > > > > > of
> > > > > > > > > AskTimeoutException)
> > > > > > > > >
> > > > > > > > > These may also need to be resolved in 1.10.1.
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Hequn
> > > > > > > > >
> > > > > > > > > [1]
> > > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com>
> > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Jincheng,
> > > > > > > > > >
> > > > > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > > > > >
> > > > > > > > > > Best Regards,
> > > > > > > > > > Yu
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > > sunjincheng121@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Thanks for bring up the discussion Yu. I would like to
> > give
> > > > you a
> > > > > > > > hand
> > > > > > > > > at
> > > > > > > > > > > the last stage when the RC is finished.(If you need)
> :)
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Jincheng
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > > > > >
> > > > > > > > > > > > Hi All,
> > > > > > > > > > > >
> > > > > > > > > > > > It has been almost one month since we released Flink
> > > > 1.10.0. We
> > > > > > > > > already
> > > > > > > > > > > > have more than 40 resolved improvements/bugs in the
> > > > > > release-1.10
> > > > > > > > > > branch,
> > > > > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > > > > >
> > > > > > > > > > > > Most noticeable fixes are:
> > > > > > > > > > > >
> > > > > > > > > > > > - FLINK-16241 [legal] Remove the license and notice
> > file
> > > in
> > > > > > > > > > flink-ml-lib
> > > > > > > > > > > > module
> > > > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > > flink-state-processor-api
> > > > > > > > > > > > - FLINK-16161 Statistics zero should be known in
> > > > HiveCatalog
> > > > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in
> > > TypeExtractor
> > > > > > when
> > > > > > > > > > mapping
> > > > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if running
> in
> > > > blink
> > > > > > > > planner
> > > > > > > > > > > > - FLINK-16139 Co-location constraints are not reset
> on
> > > task
> > > > > > > > recovery
> > > > > > > > > in
> > > > > > > > > > > > DefaultScheduler
> > > > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql
> > casuing
> > > > > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > > > > >
> > > > > > > > > > > > Furthermore, I think the following issues should be
> > > merged
> > > > > > before
> > > > > > > > > > 1.10.1
> > > > > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > > > > >
> > > > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error
> on
> > > > > > repeated
> > > > > > > > job
> > > > > > > > > > > > submission
> > > > > > > > > > > > - FLINK-16406 Increase default value for JVM
> Metaspace
> > to
> > > > > > > minimise
> > > > > > > > > its
> > > > > > > > > > > > OutOfMemoryError
> > > > > > > > > > > > - FLINK-16047 Blink planner produces wrong aggregate
> > > > results
> > > > > > with
> > > > > > > > > state
> > > > > > > > > > > > clean up
> > > > > > > > > > > > - FLINK-16070 Blink planner can not extract correct
> > > unique
> > > > key
> > > > > > > for
> > > > > > > > > > > > UpsertStreamTableSink
> > > > > > > > > > > >
> > > > > > > > > > > > I would volunteer as the release manager and kick off
> > the
> > > > > > release
> > > > > > > > > > process
> > > > > > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > > > > > >
> > > > > > > > > > > > If there are any concerns or missing blocker issues
> > need
> > > > to be
> > > > > > > > fixed
> > > > > > > > > in
> > > > > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > > > > >
> > > > > > > > > > > > Best Regards,
> > > > > > > > > > > > Yu
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> > >
> > > --
> > > Best, Jingsong Lee
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the suggestion Andrey! I've added 1.10.1 into FLINK-16225 fix
versions and promoted its priority to Critical. Will also watch the
progress of FLINK-16108/FLINK-16408.
Best Regards,
Yu
On Wed, 11 Mar 2020 at 18:18, Andrey Zagrebin <az...@apache.org> wrote:
> Hi Yu,
>
> Thanks for kicking off the 1.10.1 release discussion!
>
> Apart from
> - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> OutOfMemoryError
> which should be merged soon
>
> I think we should also try to get in the following issues:
>
> - [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error in
> TaskManager
> This should solve the Metaspace problem even in a better way because OOM
> failure should point users to the docs immediately
>
> - [FLINK-16408] Bind user code class loader to lifetime of a slot
> This should give a better protection against class loading leaks
>
> - [FLINK-16018] Improve error reporting when submitting batch job (instead
> of AskTimeoutException)
> This problem has recently happened for multiple users
>
> Best,
> Andrey
>
>
> On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com>
> wrote:
>
> > Thanks for driving. Yu. +1 for starting the 1.10.1 release.
> >
> > Some issues are very important, Users are looking forward to them.
> >
> > Best,
> > Jingsong Lee
> >
> > On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com> wrote:
> >
> > > Thanks for driving this release, Yu!
> > >
> > > +1 for starting the 1.10.1 release cycle.
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <to...@gmail.com>
> > > wrote:
> > > >
> > > > Yu,
> > > > Thanks for the explanation.
> > > > I've no concerns. I was just trying to get some inputs for
> prioritizing
> > > > tasks on my side, and ~1month sounds good to me.
> > > >
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> > > >
> > > > > bq. what is the time plan for 1.10.1?
> > > > >
> > > > > According to the history, the first patch release of a major
> version
> > > will
> > > > > take ~1month from discussion started, depending on the speed of
> > blocker
> > > > > issue resolving:
> > > > > * 1.8.1: started discussion on May 28th [1], released on Jul 3rd
> [2]
> > > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th
> [4]
> > > > >
> > > > > We won't rush to match the history of course, but could use it as a
> > > > > reference. And please feel free to let me know if any concerns
> > Xintong.
> > > > > Thanks.
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > > [2]
> > > > >
> > > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > > [3]
> > > > >
> > > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > > [4]
> > > > >
> > > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > > >
> > > > >
> > > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Thanks Yu, for the kick off and volunteering to be the release
> > > manager.
> > > > > >
> > > > > > +1 for the proposal.
> > > > > >
> > > > > >
> > > > > > One quick question, what is the time plan for 1.10.1?
> > > > > >
> > > > > >
> > > > > > Thank you~
> > > > > >
> > > > > > Xintong Song
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > > <wa...@aliyun.com.invalid> wrote:
> > > > > >
> > > > > > > Thanks for driving this release, Yu!
> > > > > > > +1 on my side
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > > Zhijiang
> > > > > > >
> > > > > > >
> > > > > > >
> > ------------------------------------------------------------------
> > > > > > > From:Yu Li <ca...@gmail.com>
> > > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > > To:dev <de...@flink.apache.org>
> > > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > > >
> > > > > > > Thanks for the supplement Hequn. Yes will also keep an eye on
> > these
> > > > > > > existing blocker issues.
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Yu
> > > > > > >
> > > > > > >
> > > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org>
> > > wrote:
> > > > > > >
> > > > > > > > Hi Yu,
> > > > > > > >
> > > > > > > > Thanks a lot for raising the discussion and volunteer as the
> > > release
> > > > > > > > manager!
> > > > > > > >
> > > > > > > > I found there are some other issues[1] which are marked as a
> > > blocker:
> > > > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > > > - FLINK-16262 Class loader problem with
> > > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > > > > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException
> when
> > > use
> > > > > > > > flink-sql-connector-elasticsearch7
> > > > > > > > - FLINK-16018 Improve error reporting when submitting batch
> job
> > > > > > (instead
> > > > > > > of
> > > > > > > > AskTimeoutException)
> > > > > > > >
> > > > > > > > These may also need to be resolved in 1.10.1.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Hequn
> > > > > > > >
> > > > > > > > [1]
> > > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com>
> > wrote:
> > > > > > > >
> > > > > > > > > Hi Jincheng,
> > > > > > > > >
> > > > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Yu
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > > sunjincheng121@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for bring up the discussion Yu. I would like to
> give
> > > you a
> > > > > > > hand
> > > > > > > > at
> > > > > > > > > > the last stage when the RC is finished.(If you need) :)
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Jincheng
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > > > >
> > > > > > > > > > > Hi All,
> > > > > > > > > > >
> > > > > > > > > > > It has been almost one month since we released Flink
> > > 1.10.0. We
> > > > > > > > already
> > > > > > > > > > > have more than 40 resolved improvements/bugs in the
> > > > > release-1.10
> > > > > > > > > branch,
> > > > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > > > >
> > > > > > > > > > > Most noticeable fixes are:
> > > > > > > > > > >
> > > > > > > > > > > - FLINK-16241 [legal] Remove the license and notice
> file
> > in
> > > > > > > > > flink-ml-lib
> > > > > > > > > > > module
> > > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > > > flink-state-processor-api
> > > > > > > > > > > - FLINK-16161 Statistics zero should be known in
> > > HiveCatalog
> > > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in
> > TypeExtractor
> > > > > when
> > > > > > > > > mapping
> > > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if running in
> > > blink
> > > > > > > planner
> > > > > > > > > > > - FLINK-16139 Co-location constraints are not reset on
> > task
> > > > > > > recovery
> > > > > > > > in
> > > > > > > > > > > DefaultScheduler
> > > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql
> casuing
> > > > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > > > >
> > > > > > > > > > > Furthermore, I think the following issues should be
> > merged
> > > > > before
> > > > > > > > > 1.10.1
> > > > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > > > >
> > > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > > > repeated
> > > > > > > job
> > > > > > > > > > > submission
> > > > > > > > > > > - FLINK-16406 Increase default value for JVM Metaspace
> to
> > > > > > minimise
> > > > > > > > its
> > > > > > > > > > > OutOfMemoryError
> > > > > > > > > > > - FLINK-16047 Blink planner produces wrong aggregate
> > > results
> > > > > with
> > > > > > > > state
> > > > > > > > > > > clean up
> > > > > > > > > > > - FLINK-16070 Blink planner can not extract correct
> > unique
> > > key
> > > > > > for
> > > > > > > > > > > UpsertStreamTableSink
> > > > > > > > > > >
> > > > > > > > > > > I would volunteer as the release manager and kick off
> the
> > > > > release
> > > > > > > > > process
> > > > > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > > > > >
> > > > > > > > > > > If there are any concerns or missing blocker issues
> need
> > > to be
> > > > > > > fixed
> > > > > > > > in
> > > > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > > > >
> > > > > > > > > > > Best Regards,
> > > > > > > > > > > Yu
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Andrey Zagrebin <az...@apache.org>.
Hi Yu,
Thanks for kicking off the 1.10.1 release discussion!
Apart from
- FLINK-16406 Increase default value for JVM Metaspace to minimise its
OutOfMemoryError
which should be merged soon
I think we should also try to get in the following issues:
- [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error in
TaskManager
This should solve the Metaspace problem even in a better way because OOM
failure should point users to the docs immediately
- [FLINK-16408] Bind user code class loader to lifetime of a slot
This should give a better protection against class loading leaks
- [FLINK-16018] Improve error reporting when submitting batch job (instead
of AskTimeoutException)
This problem has recently happened for multiple users
Best,
Andrey
On Wed, Mar 11, 2020 at 8:46 AM Jingsong Li <ji...@gmail.com> wrote:
> Thanks for driving. Yu. +1 for starting the 1.10.1 release.
>
> Some issues are very important, Users are looking forward to them.
>
> Best,
> Jingsong Lee
>
> On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com> wrote:
>
> > Thanks for driving this release, Yu!
> >
> > +1 for starting the 1.10.1 release cycle.
> >
> > Best,
> > Yangze Guo
> >
> > On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <to...@gmail.com>
> > wrote:
> > >
> > > Yu,
> > > Thanks for the explanation.
> > > I've no concerns. I was just trying to get some inputs for prioritizing
> > > tasks on my side, and ~1month sounds good to me.
> > >
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > >
> > > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> > >
> > > > bq. what is the time plan for 1.10.1?
> > > >
> > > > According to the history, the first patch release of a major version
> > will
> > > > take ~1month from discussion started, depending on the speed of
> blocker
> > > > issue resolving:
> > > > * 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
> > > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th [4]
> > > >
> > > > We won't rush to match the history of course, but could use it as a
> > > > reference. And please feel free to let me know if any concerns
> Xintong.
> > > > Thanks.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > > [1]
> > > >
> > > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > > [2]
> > > >
> > > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > > [3]
> > > >
> > > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > > [4]
> > > >
> > > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > > >
> > > >
> > > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com>
> > wrote:
> > > >
> > > > > Thanks Yu, for the kick off and volunteering to be the release
> > manager.
> > > > >
> > > > > +1 for the proposal.
> > > > >
> > > > >
> > > > > One quick question, what is the time plan for 1.10.1?
> > > > >
> > > > >
> > > > > Thank you~
> > > > >
> > > > > Xintong Song
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > > <wa...@aliyun.com.invalid> wrote:
> > > > >
> > > > > > Thanks for driving this release, Yu!
> > > > > > +1 on my side
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > > Zhijiang
> > > > > >
> > > > > >
> > > > > >
> ------------------------------------------------------------------
> > > > > > From:Yu Li <ca...@gmail.com>
> > > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > > To:dev <de...@flink.apache.org>
> > > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > > >
> > > > > > Thanks for the supplement Hequn. Yes will also keep an eye on
> these
> > > > > > existing blocker issues.
> > > > > >
> > > > > > Best Regards,
> > > > > > Yu
> > > > > >
> > > > > >
> > > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org>
> > wrote:
> > > > > >
> > > > > > > Hi Yu,
> > > > > > >
> > > > > > > Thanks a lot for raising the discussion and volunteer as the
> > release
> > > > > > > manager!
> > > > > > >
> > > > > > > I found there are some other issues[1] which are marked as a
> > blocker:
> > > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > > - FLINK-16262 Class loader problem with
> > > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > > > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when
> > use
> > > > > > > flink-sql-connector-elasticsearch7
> > > > > > > - FLINK-16018 Improve error reporting when submitting batch job
> > > > > (instead
> > > > > > of
> > > > > > > AskTimeoutException)
> > > > > > >
> > > > > > > These may also need to be resolved in 1.10.1.
> > > > > > >
> > > > > > > Best,
> > > > > > > Hequn
> > > > > > >
> > > > > > > [1]
> > https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com>
> wrote:
> > > > > > >
> > > > > > > > Hi Jincheng,
> > > > > > > >
> > > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Yu
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > > sunjincheng121@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks for bring up the discussion Yu. I would like to give
> > you a
> > > > > > hand
> > > > > > > at
> > > > > > > > > the last stage when the RC is finished.(If you need) :)
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Jincheng
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > > >
> > > > > > > > > > Hi All,
> > > > > > > > > >
> > > > > > > > > > It has been almost one month since we released Flink
> > 1.10.0. We
> > > > > > > already
> > > > > > > > > > have more than 40 resolved improvements/bugs in the
> > > > release-1.10
> > > > > > > > branch,
> > > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > > >
> > > > > > > > > > Most noticeable fixes are:
> > > > > > > > > >
> > > > > > > > > > - FLINK-16241 [legal] Remove the license and notice file
> in
> > > > > > > > flink-ml-lib
> > > > > > > > > > module
> > > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > > flink-state-processor-api
> > > > > > > > > > - FLINK-16161 Statistics zero should be known in
> > HiveCatalog
> > > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in
> TypeExtractor
> > > > when
> > > > > > > > mapping
> > > > > > > > > > - FLINK-16108 StreamSQLExample is failed if running in
> > blink
> > > > > > planner
> > > > > > > > > > - FLINK-16139 Co-location constraints are not reset on
> task
> > > > > > recovery
> > > > > > > in
> > > > > > > > > > DefaultScheduler
> > > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > > >
> > > > > > > > > > Furthermore, I think the following issues should be
> merged
> > > > before
> > > > > > > > 1.10.1
> > > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > > >
> > > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > > repeated
> > > > > > job
> > > > > > > > > > submission
> > > > > > > > > > - FLINK-16406 Increase default value for JVM Metaspace to
> > > > > minimise
> > > > > > > its
> > > > > > > > > > OutOfMemoryError
> > > > > > > > > > - FLINK-16047 Blink planner produces wrong aggregate
> > results
> > > > with
> > > > > > > state
> > > > > > > > > > clean up
> > > > > > > > > > - FLINK-16070 Blink planner can not extract correct
> unique
> > key
> > > > > for
> > > > > > > > > > UpsertStreamTableSink
> > > > > > > > > >
> > > > > > > > > > I would volunteer as the release manager and kick off the
> > > > release
> > > > > > > > process
> > > > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > > > >
> > > > > > > > > > If there are any concerns or missing blocker issues need
> > to be
> > > > > > fixed
> > > > > > > in
> > > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > > >
> > > > > > > > > > Best Regards,
> > > > > > > > > > Yu
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
>
> --
> Best, Jingsong Lee
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Jingsong Li <ji...@gmail.com>.
Thanks for driving. Yu. +1 for starting the 1.10.1 release.
Some issues are very important, Users are looking forward to them.
Best,
Jingsong Lee
On Wed, Mar 11, 2020 at 2:52 PM Yangze Guo <ka...@gmail.com> wrote:
> Thanks for driving this release, Yu!
>
> +1 for starting the 1.10.1 release cycle.
>
> Best,
> Yangze Guo
>
> On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <to...@gmail.com>
> wrote:
> >
> > Yu,
> > Thanks for the explanation.
> > I've no concerns. I was just trying to get some inputs for prioritizing
> > tasks on my side, and ~1month sounds good to me.
> >
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> >
> > > bq. what is the time plan for 1.10.1?
> > >
> > > According to the history, the first patch release of a major version
> will
> > > take ~1month from discussion started, depending on the speed of blocker
> > > issue resolving:
> > > * 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
> > > * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th [4]
> > >
> > > We won't rush to match the history of course, but could use it as a
> > > reference. And please feel free to let me know if any concerns Xintong.
> > > Thanks.
> > >
> > > Best Regards,
> > > Yu
> > >
> > > [1]
> > >
> > >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > > [2]
> > >
> > >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > > [3]
> > >
> > >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > > [4]
> > >
> > >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> > >
> > >
> > > On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com>
> wrote:
> > >
> > > > Thanks Yu, for the kick off and volunteering to be the release
> manager.
> > > >
> > > > +1 for the proposal.
> > > >
> > > >
> > > > One quick question, what is the time plan for 1.10.1?
> > > >
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > >
> > > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > > <wa...@aliyun.com.invalid> wrote:
> > > >
> > > > > Thanks for driving this release, Yu!
> > > > > +1 on my side
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Best,
> > > > > Zhijiang
> > > > >
> > > > >
> > > > > ------------------------------------------------------------------
> > > > > From:Yu Li <ca...@gmail.com>
> > > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > > To:dev <de...@flink.apache.org>
> > > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > > >
> > > > > Thanks for the supplement Hequn. Yes will also keep an eye on these
> > > > > existing blocker issues.
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > > >
> > > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org>
> wrote:
> > > > >
> > > > > > Hi Yu,
> > > > > >
> > > > > > Thanks a lot for raising the discussion and volunteer as the
> release
> > > > > > manager!
> > > > > >
> > > > > > I found there are some other issues[1] which are marked as a
> blocker:
> > > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > > - FLINK-16262 Class loader problem with
> > > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when
> use
> > > > > > flink-sql-connector-elasticsearch7
> > > > > > - FLINK-16018 Improve error reporting when submitting batch job
> > > > (instead
> > > > > of
> > > > > > AskTimeoutException)
> > > > > >
> > > > > > These may also need to be resolved in 1.10.1.
> > > > > >
> > > > > > Best,
> > > > > > Hequn
> > > > > >
> > > > > > [1]
> https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > > >
> > > > > >
> > > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> > > > > >
> > > > > > > Hi Jincheng,
> > > > > > >
> > > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Yu
> > > > > > >
> > > > > > >
> > > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > > sunjincheng121@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks for bring up the discussion Yu. I would like to give
> you a
> > > > > hand
> > > > > > at
> > > > > > > > the last stage when the RC is finished.(If you need) :)
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Jincheng
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > > >
> > > > > > > > > Hi All,
> > > > > > > > >
> > > > > > > > > It has been almost one month since we released Flink
> 1.10.0. We
> > > > > > already
> > > > > > > > > have more than 40 resolved improvements/bugs in the
> > > release-1.10
> > > > > > > branch,
> > > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > > >
> > > > > > > > > Most noticeable fixes are:
> > > > > > > > >
> > > > > > > > > - FLINK-16241 [legal] Remove the license and notice file in
> > > > > > > flink-ml-lib
> > > > > > > > > module
> > > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > > flink-state-processor-api
> > > > > > > > > - FLINK-16161 Statistics zero should be known in
> HiveCatalog
> > > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor
> > > when
> > > > > > > mapping
> > > > > > > > > - FLINK-16108 StreamSQLExample is failed if running in
> blink
> > > > > planner
> > > > > > > > > - FLINK-16139 Co-location constraints are not reset on task
> > > > > recovery
> > > > > > in
> > > > > > > > > DefaultScheduler
> > > > > > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > > > > > ValidationException: SQL validation failed
> > > > > > > > >
> > > > > > > > > Furthermore, I think the following issues should be merged
> > > before
> > > > > > > 1.10.1
> > > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > > >
> > > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on
> > > repeated
> > > > > job
> > > > > > > > > submission
> > > > > > > > > - FLINK-16406 Increase default value for JVM Metaspace to
> > > > minimise
> > > > > > its
> > > > > > > > > OutOfMemoryError
> > > > > > > > > - FLINK-16047 Blink planner produces wrong aggregate
> results
> > > with
> > > > > > state
> > > > > > > > > clean up
> > > > > > > > > - FLINK-16070 Blink planner can not extract correct unique
> key
> > > > for
> > > > > > > > > UpsertStreamTableSink
> > > > > > > > >
> > > > > > > > > I would volunteer as the release manager and kick off the
> > > release
> > > > > > > process
> > > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > > >
> > > > > > > > > If there are any concerns or missing blocker issues need
> to be
> > > > > fixed
> > > > > > in
> > > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > > >
> > > > > > > > > Best Regards,
> > > > > > > > > Yu
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
--
Best, Jingsong Lee
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yangze Guo <ka...@gmail.com>.
Thanks for driving this release, Yu!
+1 for starting the 1.10.1 release cycle.
Best,
Yangze Guo
On Wed, Mar 11, 2020 at 1:42 PM Xintong Song <to...@gmail.com> wrote:
>
> Yu,
> Thanks for the explanation.
> I've no concerns. I was just trying to get some inputs for prioritizing
> tasks on my side, and ~1month sounds good to me.
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
>
> > bq. what is the time plan for 1.10.1?
> >
> > According to the history, the first patch release of a major version will
> > take ~1month from discussion started, depending on the speed of blocker
> > issue resolving:
> > * 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
> > * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th [4]
> >
> > We won't rush to match the history of course, but could use it as a
> > reference. And please feel free to let me know if any concerns Xintong.
> > Thanks.
> >
> > Best Regards,
> > Yu
> >
> > [1]
> >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> > [2]
> >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> > [3]
> >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> > [4]
> >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
> >
> >
> > On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com> wrote:
> >
> > > Thanks Yu, for the kick off and volunteering to be the release manager.
> > >
> > > +1 for the proposal.
> > >
> > >
> > > One quick question, what is the time plan for 1.10.1?
> > >
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > >
> > > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > > <wa...@aliyun.com.invalid> wrote:
> > >
> > > > Thanks for driving this release, Yu!
> > > > +1 on my side
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Best,
> > > > Zhijiang
> > > >
> > > >
> > > > ------------------------------------------------------------------
> > > > From:Yu Li <ca...@gmail.com>
> > > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > > To:dev <de...@flink.apache.org>
> > > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > > >
> > > > Thanks for the supplement Hequn. Yes will also keep an eye on these
> > > > existing blocker issues.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
> > > >
> > > > > Hi Yu,
> > > > >
> > > > > Thanks a lot for raising the discussion and volunteer as the release
> > > > > manager!
> > > > >
> > > > > I found there are some other issues[1] which are marked as a blocker:
> > > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > > - FLINK-16262 Class loader problem with
> > > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> > > > > flink-sql-connector-elasticsearch7
> > > > > - FLINK-16018 Improve error reporting when submitting batch job
> > > (instead
> > > > of
> > > > > AskTimeoutException)
> > > > >
> > > > > These may also need to be resolved in 1.10.1.
> > > > >
> > > > > Best,
> > > > > Hequn
> > > > >
> > > > > [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > > >
> > > > >
> > > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> > > > >
> > > > > > Hi Jincheng,
> > > > > >
> > > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > > >
> > > > > > Best Regards,
> > > > > > Yu
> > > > > >
> > > > > >
> > > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> > sunjincheng121@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks for bring up the discussion Yu. I would like to give you a
> > > > hand
> > > > > at
> > > > > > > the last stage when the RC is finished.(If you need) :)
> > > > > > >
> > > > > > > Best,
> > > > > > > Jincheng
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > > >
> > > > > > > > Hi All,
> > > > > > > >
> > > > > > > > It has been almost one month since we released Flink 1.10.0. We
> > > > > already
> > > > > > > > have more than 40 resolved improvements/bugs in the
> > release-1.10
> > > > > > branch,
> > > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > > >
> > > > > > > > Most noticeable fixes are:
> > > > > > > >
> > > > > > > > - FLINK-16241 [legal] Remove the license and notice file in
> > > > > > flink-ml-lib
> > > > > > > > module
> > > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > > flink-state-processor-api
> > > > > > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor
> > when
> > > > > > mapping
> > > > > > > > - FLINK-16108 StreamSQLExample is failed if running in blink
> > > > planner
> > > > > > > > - FLINK-16139 Co-location constraints are not reset on task
> > > > recovery
> > > > > in
> > > > > > > > DefaultScheduler
> > > > > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > > > > ValidationException: SQL validation failed
> > > > > > > >
> > > > > > > > Furthermore, I think the following issues should be merged
> > before
> > > > > > 1.10.1
> > > > > > > > release (especially the Metaspace OOM issue):
> > > > > > > >
> > > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on
> > repeated
> > > > job
> > > > > > > > submission
> > > > > > > > - FLINK-16406 Increase default value for JVM Metaspace to
> > > minimise
> > > > > its
> > > > > > > > OutOfMemoryError
> > > > > > > > - FLINK-16047 Blink planner produces wrong aggregate results
> > with
> > > > > state
> > > > > > > > clean up
> > > > > > > > - FLINK-16070 Blink planner can not extract correct unique key
> > > for
> > > > > > > > UpsertStreamTableSink
> > > > > > > >
> > > > > > > > I would volunteer as the release manager and kick off the
> > release
> > > > > > process
> > > > > > > > once blocker issues are merged. What do you think?
> > > > > > > >
> > > > > > > > If there are any concerns or missing blocker issues need to be
> > > > fixed
> > > > > in
> > > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > > >
> > > > > > > > Best Regards,
> > > > > > > > Yu
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Xintong Song <to...@gmail.com>.
Yu,
Thanks for the explanation.
I've no concerns. I was just trying to get some inputs for prioritizing
tasks on my side, and ~1month sounds good to me.
Thank you~
Xintong Song
On Wed, Mar 11, 2020 at 12:15 PM Yu Li <ca...@gmail.com> wrote:
> bq. what is the time plan for 1.10.1?
>
> According to the history, the first patch release of a major version will
> take ~1month from discussion started, depending on the speed of blocker
> issue resolving:
> * 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
> * 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th [4]
>
> We won't rush to match the history of course, but could use it as a
> reference. And please feel free to let me know if any concerns Xintong.
> Thanks.
>
> Best Regards,
> Yu
>
> [1]
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
> [2]
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
> [3]
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
> [4]
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
>
>
> On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com> wrote:
>
> > Thanks Yu, for the kick off and volunteering to be the release manager.
> >
> > +1 for the proposal.
> >
> >
> > One quick question, what is the time plan for 1.10.1?
> >
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> > <wa...@aliyun.com.invalid> wrote:
> >
> > > Thanks for driving this release, Yu!
> > > +1 on my side
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Best,
> > > Zhijiang
> > >
> > >
> > > ------------------------------------------------------------------
> > > From:Yu Li <ca...@gmail.com>
> > > Send Time:2020 Mar. 10 (Tue.) 20:25
> > > To:dev <de...@flink.apache.org>
> > > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> > >
> > > Thanks for the supplement Hequn. Yes will also keep an eye on these
> > > existing blocker issues.
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
> > >
> > > > Hi Yu,
> > > >
> > > > Thanks a lot for raising the discussion and volunteer as the release
> > > > manager!
> > > >
> > > > I found there are some other issues[1] which are marked as a blocker:
> > > > - FLINK-16454 Update the copyright year in NOTICE files
> > > > - FLINK-16262 Class loader problem with
> > > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> > > > flink-sql-connector-elasticsearch7
> > > > - FLINK-16018 Improve error reporting when submitting batch job
> > (instead
> > > of
> > > > AskTimeoutException)
> > > >
> > > > These may also need to be resolved in 1.10.1.
> > > >
> > > > Best,
> > > > Hequn
> > > >
> > > > [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > > >
> > > >
> > > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> > > >
> > > > > Hi Jincheng,
> > > > >
> > > > > Yes, your help would be very helpful. Thanks a lot!
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > > >
> > > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <
> sunjincheng121@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Thanks for bring up the discussion Yu. I would like to give you a
> > > hand
> > > > at
> > > > > > the last stage when the RC is finished.(If you need) :)
> > > > > >
> > > > > > Best,
> > > > > > Jincheng
> > > > > >
> > > > > >
> > > > > >
> > > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > > It has been almost one month since we released Flink 1.10.0. We
> > > > already
> > > > > > > have more than 40 resolved improvements/bugs in the
> release-1.10
> > > > > branch,
> > > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > > >
> > > > > > > Most noticeable fixes are:
> > > > > > >
> > > > > > > - FLINK-16241 [legal] Remove the license and notice file in
> > > > > flink-ml-lib
> > > > > > > module
> > > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > > flink-state-processor-api
> > > > > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor
> when
> > > > > mapping
> > > > > > > - FLINK-16108 StreamSQLExample is failed if running in blink
> > > planner
> > > > > > > - FLINK-16139 Co-location constraints are not reset on task
> > > recovery
> > > > in
> > > > > > > DefaultScheduler
> > > > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > > > ValidationException: SQL validation failed
> > > > > > >
> > > > > > > Furthermore, I think the following issues should be merged
> before
> > > > > 1.10.1
> > > > > > > release (especially the Metaspace OOM issue):
> > > > > > >
> > > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on
> repeated
> > > job
> > > > > > > submission
> > > > > > > - FLINK-16406 Increase default value for JVM Metaspace to
> > minimise
> > > > its
> > > > > > > OutOfMemoryError
> > > > > > > - FLINK-16047 Blink planner produces wrong aggregate results
> with
> > > > state
> > > > > > > clean up
> > > > > > > - FLINK-16070 Blink planner can not extract correct unique key
> > for
> > > > > > > UpsertStreamTableSink
> > > > > > >
> > > > > > > I would volunteer as the release manager and kick off the
> release
> > > > > process
> > > > > > > once blocker issues are merged. What do you think?
> > > > > > >
> > > > > > > If there are any concerns or missing blocker issues need to be
> > > fixed
> > > > in
> > > > > > > 1.10.1, please let me know. Thanks.
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Yu
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
bq. what is the time plan for 1.10.1?
According to the history, the first patch release of a major version will
take ~1month from discussion started, depending on the speed of blocker
issue resolving:
* 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
* 1.9.1: started discussion on Sep 23rd [3], released on Oct 19th [4]
We won't rush to match the history of course, but could use it as a
reference. And please feel free to let me know if any concerns Xintong.
Thanks.
Best Regards,
Yu
[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-1-td29154.html
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-8-1-released-td30124.html
[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-9-1-td33343.html
[4]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Apache-Flink-1-9-1-released-td34170.html
On Wed, 11 Mar 2020 at 11:54, Xintong Song <to...@gmail.com> wrote:
> Thanks Yu, for the kick off and volunteering to be the release manager.
>
> +1 for the proposal.
>
>
> One quick question, what is the time plan for 1.10.1?
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
> <wa...@aliyun.com.invalid> wrote:
>
> > Thanks for driving this release, Yu!
> > +1 on my side
> >
> >
> >
> >
> >
> >
> >
> > Best,
> > Zhijiang
> >
> >
> > ------------------------------------------------------------------
> > From:Yu Li <ca...@gmail.com>
> > Send Time:2020 Mar. 10 (Tue.) 20:25
> > To:dev <de...@flink.apache.org>
> > Subject:Re: [DISCUSS] Releasing Flink 1.10.1
> >
> > Thanks for the supplement Hequn. Yes will also keep an eye on these
> > existing blocker issues.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
> >
> > > Hi Yu,
> > >
> > > Thanks a lot for raising the discussion and volunteer as the release
> > > manager!
> > >
> > > I found there are some other issues[1] which are marked as a blocker:
> > > - FLINK-16454 Update the copyright year in NOTICE files
> > > - FLINK-16262 Class loader problem with
> > > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> > > flink-sql-connector-elasticsearch7
> > > - FLINK-16018 Improve error reporting when submitting batch job
> (instead
> > of
> > > AskTimeoutException)
> > >
> > > These may also need to be resolved in 1.10.1.
> > >
> > > Best,
> > > Hequn
> > >
> > > [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
> > >
> > >
> > > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> > >
> > > > Hi Jincheng,
> > > >
> > > > Yes, your help would be very helpful. Thanks a lot!
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > > >
> > > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <sunjincheng121@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Thanks for bring up the discussion Yu. I would like to give you a
> > hand
> > > at
> > > > > the last stage when the RC is finished.(If you need) :)
> > > > >
> > > > > Best,
> > > > > Jincheng
> > > > >
> > > > >
> > > > >
> > > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > It has been almost one month since we released Flink 1.10.0. We
> > > already
> > > > > > have more than 40 resolved improvements/bugs in the release-1.10
> > > > branch,
> > > > > > and I propose to start the 1.10.1 release cycle.
> > > > > >
> > > > > > Most noticeable fixes are:
> > > > > >
> > > > > > - FLINK-16241 [legal] Remove the license and notice file in
> > > > flink-ml-lib
> > > > > > module
> > > > > > - FLINK-16313 Fix RocksDB resource leak in
> > flink-state-processor-api
> > > > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when
> > > > mapping
> > > > > > - FLINK-16108 StreamSQLExample is failed if running in blink
> > planner
> > > > > > - FLINK-16139 Co-location constraints are not reset on task
> > recovery
> > > in
> > > > > > DefaultScheduler
> > > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > > ValidationException: SQL validation failed
> > > > > >
> > > > > > Furthermore, I think the following issues should be merged before
> > > > 1.10.1
> > > > > > release (especially the Metaspace OOM issue):
> > > > > >
> > > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> > job
> > > > > > submission
> > > > > > - FLINK-16406 Increase default value for JVM Metaspace to
> minimise
> > > its
> > > > > > OutOfMemoryError
> > > > > > - FLINK-16047 Blink planner produces wrong aggregate results with
> > > state
> > > > > > clean up
> > > > > > - FLINK-16070 Blink planner can not extract correct unique key
> for
> > > > > > UpsertStreamTableSink
> > > > > >
> > > > > > I would volunteer as the release manager and kick off the release
> > > > process
> > > > > > once blocker issues are merged. What do you think?
> > > > > >
> > > > > > If there are any concerns or missing blocker issues need to be
> > fixed
> > > in
> > > > > > 1.10.1, please let me know. Thanks.
> > > > > >
> > > > > > Best Regards,
> > > > > > Yu
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Xintong Song <to...@gmail.com>.
Thanks Yu, for the kick off and volunteering to be the release manager.
+1 for the proposal.
One quick question, what is the time plan for 1.10.1?
Thank you~
Xintong Song
On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
<wa...@aliyun.com.invalid> wrote:
> Thanks for driving this release, Yu!
> +1 on my side
>
>
>
>
>
>
>
> Best,
> Zhijiang
>
>
> ------------------------------------------------------------------
> From:Yu Li <ca...@gmail.com>
> Send Time:2020 Mar. 10 (Tue.) 20:25
> To:dev <de...@flink.apache.org>
> Subject:Re: [DISCUSS] Releasing Flink 1.10.1
>
> Thanks for the supplement Hequn. Yes will also keep an eye on these
> existing blocker issues.
>
> Best Regards,
> Yu
>
>
> On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
>
> > Hi Yu,
> >
> > Thanks a lot for raising the discussion and volunteer as the release
> > manager!
> >
> > I found there are some other issues[1] which are marked as a blocker:
> > - FLINK-16454 Update the copyright year in NOTICE files
> > - FLINK-16262 Class loader problem with
> > FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> > - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> > flink-sql-connector-elasticsearch7
> > - FLINK-16018 Improve error reporting when submitting batch job (instead
> of
> > AskTimeoutException)
> >
> > These may also need to be resolved in 1.10.1.
> >
> > Best,
> > Hequn
> >
> > [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
> >
> >
> > On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> >
> > > Hi Jincheng,
> > >
> > > Yes, your help would be very helpful. Thanks a lot!
> > >
> > > Best Regards,
> > > Yu
> > >
> > >
> > > On Tue, 10 Mar 2020 at 18:24, jincheng sun <su...@gmail.com>
> > > wrote:
> > >
> > > > Thanks for bring up the discussion Yu. I would like to give you a
> hand
> > at
> > > > the last stage when the RC is finished.(If you need) :)
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > > >
> > > >
> > > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > > >
> > > > > Hi All,
> > > > >
> > > > > It has been almost one month since we released Flink 1.10.0. We
> > already
> > > > > have more than 40 resolved improvements/bugs in the release-1.10
> > > branch,
> > > > > and I propose to start the 1.10.1 release cycle.
> > > > >
> > > > > Most noticeable fixes are:
> > > > >
> > > > > - FLINK-16241 [legal] Remove the license and notice file in
> > > flink-ml-lib
> > > > > module
> > > > > - FLINK-16313 Fix RocksDB resource leak in
> flink-state-processor-api
> > > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when
> > > mapping
> > > > > - FLINK-16108 StreamSQLExample is failed if running in blink
> planner
> > > > > - FLINK-16139 Co-location constraints are not reset on task
> recovery
> > in
> > > > > DefaultScheduler
> > > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > > ValidationException: SQL validation failed
> > > > >
> > > > > Furthermore, I think the following issues should be merged before
> > > 1.10.1
> > > > > release (especially the Metaspace OOM issue):
> > > > >
> > > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated
> job
> > > > > submission
> > > > > - FLINK-16406 Increase default value for JVM Metaspace to minimise
> > its
> > > > > OutOfMemoryError
> > > > > - FLINK-16047 Blink planner produces wrong aggregate results with
> > state
> > > > > clean up
> > > > > - FLINK-16070 Blink planner can not extract correct unique key for
> > > > > UpsertStreamTableSink
> > > > >
> > > > > I would volunteer as the release manager and kick off the release
> > > process
> > > > > once blocker issues are merged. What do you think?
> > > > >
> > > > > If there are any concerns or missing blocker issues need to be
> fixed
> > in
> > > > > 1.10.1, please let me know. Thanks.
> > > > >
> > > > > Best Regards,
> > > > > Yu
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Zhijiang <wa...@aliyun.com.INVALID>.
Thanks for driving this release, Yu!
+1 on my side
Best,
Zhijiang
------------------------------------------------------------------
From:Yu Li <ca...@gmail.com>
Send Time:2020 Mar. 10 (Tue.) 20:25
To:dev <de...@flink.apache.org>
Subject:Re: [DISCUSS] Releasing Flink 1.10.1
Thanks for the supplement Hequn. Yes will also keep an eye on these
existing blocker issues.
Best Regards,
Yu
On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
> Hi Yu,
>
> Thanks a lot for raising the discussion and volunteer as the release
> manager!
>
> I found there are some other issues[1] which are marked as a blocker:
> - FLINK-16454 Update the copyright year in NOTICE files
> - FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> flink-sql-connector-elasticsearch7
> - FLINK-16018 Improve error reporting when submitting batch job (instead of
> AskTimeoutException)
>
> These may also need to be resolved in 1.10.1.
>
> Best,
> Hequn
>
> [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
>
>
> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
>
> > Hi Jincheng,
> >
> > Yes, your help would be very helpful. Thanks a lot!
> >
> > Best Regards,
> > Yu
> >
> >
> > On Tue, 10 Mar 2020 at 18:24, jincheng sun <su...@gmail.com>
> > wrote:
> >
> > > Thanks for bring up the discussion Yu. I would like to give you a hand
> at
> > > the last stage when the RC is finished.(If you need) :)
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > >
> > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > >
> > > > Hi All,
> > > >
> > > > It has been almost one month since we released Flink 1.10.0. We
> already
> > > > have more than 40 resolved improvements/bugs in the release-1.10
> > branch,
> > > > and I propose to start the 1.10.1 release cycle.
> > > >
> > > > Most noticeable fixes are:
> > > >
> > > > - FLINK-16241 [legal] Remove the license and notice file in
> > flink-ml-lib
> > > > module
> > > > - FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
> > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when
> > mapping
> > > > - FLINK-16108 StreamSQLExample is failed if running in blink planner
> > > > - FLINK-16139 Co-location constraints are not reset on task recovery
> in
> > > > DefaultScheduler
> > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > ValidationException: SQL validation failed
> > > >
> > > > Furthermore, I think the following issues should be merged before
> > 1.10.1
> > > > release (especially the Metaspace OOM issue):
> > > >
> > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
> > > > submission
> > > > - FLINK-16406 Increase default value for JVM Metaspace to minimise
> its
> > > > OutOfMemoryError
> > > > - FLINK-16047 Blink planner produces wrong aggregate results with
> state
> > > > clean up
> > > > - FLINK-16070 Blink planner can not extract correct unique key for
> > > > UpsertStreamTableSink
> > > >
> > > > I would volunteer as the release manager and kick off the release
> > process
> > > > once blocker issues are merged. What do you think?
> > > >
> > > > If there are any concerns or missing blocker issues need to be fixed
> in
> > > > 1.10.1, please let me know. Thanks.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Thanks for the supplement Hequn. Yes will also keep an eye on these
existing blocker issues.
Best Regards,
Yu
On Tue, 10 Mar 2020 at 19:10, Hequn Cheng <he...@apache.org> wrote:
> Hi Yu,
>
> Thanks a lot for raising the discussion and volunteer as the release
> manager!
>
> I found there are some other issues[1] which are marked as a blocker:
> - FLINK-16454 Update the copyright year in NOTICE files
> - FLINK-16262 Class loader problem with
> FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
> - FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
> flink-sql-connector-elasticsearch7
> - FLINK-16018 Improve error reporting when submitting batch job (instead of
> AskTimeoutException)
>
> These may also need to be resolved in 1.10.1.
>
> Best,
> Hequn
>
> [1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
>
>
> On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
>
> > Hi Jincheng,
> >
> > Yes, your help would be very helpful. Thanks a lot!
> >
> > Best Regards,
> > Yu
> >
> >
> > On Tue, 10 Mar 2020 at 18:24, jincheng sun <su...@gmail.com>
> > wrote:
> >
> > > Thanks for bring up the discussion Yu. I would like to give you a hand
> at
> > > the last stage when the RC is finished.(If you need) :)
> > >
> > > Best,
> > > Jincheng
> > >
> > >
> > >
> > > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> > >
> > > > Hi All,
> > > >
> > > > It has been almost one month since we released Flink 1.10.0. We
> already
> > > > have more than 40 resolved improvements/bugs in the release-1.10
> > branch,
> > > > and I propose to start the 1.10.1 release cycle.
> > > >
> > > > Most noticeable fixes are:
> > > >
> > > > - FLINK-16241 [legal] Remove the license and notice file in
> > flink-ml-lib
> > > > module
> > > > - FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
> > > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when
> > mapping
> > > > - FLINK-16108 StreamSQLExample is failed if running in blink planner
> > > > - FLINK-16139 Co-location constraints are not reset on task recovery
> in
> > > > DefaultScheduler
> > > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > > ValidationException: SQL validation failed
> > > >
> > > > Furthermore, I think the following issues should be merged before
> > 1.10.1
> > > > release (especially the Metaspace OOM issue):
> > > >
> > > > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
> > > > submission
> > > > - FLINK-16406 Increase default value for JVM Metaspace to minimise
> its
> > > > OutOfMemoryError
> > > > - FLINK-16047 Blink planner produces wrong aggregate results with
> state
> > > > clean up
> > > > - FLINK-16070 Blink planner can not extract correct unique key for
> > > > UpsertStreamTableSink
> > > >
> > > > I would volunteer as the release manager and kick off the release
> > process
> > > > once blocker issues are merged. What do you think?
> > > >
> > > > If there are any concerns or missing blocker issues need to be fixed
> in
> > > > 1.10.1, please let me know. Thanks.
> > > >
> > > > Best Regards,
> > > > Yu
> > > >
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Hequn Cheng <he...@apache.org>.
Hi Yu,
Thanks a lot for raising the discussion and volunteer as the release
manager!
I found there are some other issues[1] which are marked as a blocker:
- FLINK-16454 Update the copyright year in NOTICE files
- FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
- FLINK-16170 SearchTemplateRequest ClassNotFoundException when use
flink-sql-connector-elasticsearch7
- FLINK-16018 Improve error reporting when submitting batch job (instead of
AskTimeoutException)
These may also need to be resolved in 1.10.1.
Best,
Hequn
[1] https://issues.apache.org/jira/projects/FLINK/versions/12346891
On Tue, Mar 10, 2020 at 6:48 PM Yu Li <ca...@gmail.com> wrote:
> Hi Jincheng,
>
> Yes, your help would be very helpful. Thanks a lot!
>
> Best Regards,
> Yu
>
>
> On Tue, 10 Mar 2020 at 18:24, jincheng sun <su...@gmail.com>
> wrote:
>
> > Thanks for bring up the discussion Yu. I would like to give you a hand at
> > the last stage when the RC is finished.(If you need) :)
> >
> > Best,
> > Jincheng
> >
> >
> >
> > Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> >
> > > Hi All,
> > >
> > > It has been almost one month since we released Flink 1.10.0. We already
> > > have more than 40 resolved improvements/bugs in the release-1.10
> branch,
> > > and I propose to start the 1.10.1 release cycle.
> > >
> > > Most noticeable fixes are:
> > >
> > > - FLINK-16241 [legal] Remove the license and notice file in
> flink-ml-lib
> > > module
> > > - FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
> > > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when
> mapping
> > > - FLINK-16108 StreamSQLExample is failed if running in blink planner
> > > - FLINK-16139 Co-location constraints are not reset on task recovery in
> > > DefaultScheduler
> > > - FLINK-16414 Create udaf/udtf function using sql casuing
> > > ValidationException: SQL validation failed
> > >
> > > Furthermore, I think the following issues should be merged before
> 1.10.1
> > > release (especially the Metaspace OOM issue):
> > >
> > > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
> > > submission
> > > - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> > > OutOfMemoryError
> > > - FLINK-16047 Blink planner produces wrong aggregate results with state
> > > clean up
> > > - FLINK-16070 Blink planner can not extract correct unique key for
> > > UpsertStreamTableSink
> > >
> > > I would volunteer as the release manager and kick off the release
> process
> > > once blocker issues are merged. What do you think?
> > >
> > > If there are any concerns or missing blocker issues need to be fixed in
> > > 1.10.1, please let me know. Thanks.
> > >
> > > Best Regards,
> > > Yu
> > >
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by Yu Li <ca...@gmail.com>.
Hi Jincheng,
Yes, your help would be very helpful. Thanks a lot!
Best Regards,
Yu
On Tue, 10 Mar 2020 at 18:24, jincheng sun <su...@gmail.com> wrote:
> Thanks for bring up the discussion Yu. I would like to give you a hand at
> the last stage when the RC is finished.(If you need) :)
>
> Best,
> Jincheng
>
>
>
> Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
>
> > Hi All,
> >
> > It has been almost one month since we released Flink 1.10.0. We already
> > have more than 40 resolved improvements/bugs in the release-1.10 branch,
> > and I propose to start the 1.10.1 release cycle.
> >
> > Most noticeable fixes are:
> >
> > - FLINK-16241 [legal] Remove the license and notice file in flink-ml-lib
> > module
> > - FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
> > - FLINK-16161 Statistics zero should be known in HiveCatalog
> > - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when mapping
> > - FLINK-16108 StreamSQLExample is failed if running in blink planner
> > - FLINK-16139 Co-location constraints are not reset on task recovery in
> > DefaultScheduler
> > - FLINK-16414 Create udaf/udtf function using sql casuing
> > ValidationException: SQL validation failed
> >
> > Furthermore, I think the following issues should be merged before 1.10.1
> > release (especially the Metaspace OOM issue):
> >
> > - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
> > submission
> > - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> > OutOfMemoryError
> > - FLINK-16047 Blink planner produces wrong aggregate results with state
> > clean up
> > - FLINK-16070 Blink planner can not extract correct unique key for
> > UpsertStreamTableSink
> >
> > I would volunteer as the release manager and kick off the release process
> > once blocker issues are merged. What do you think?
> >
> > If there are any concerns or missing blocker issues need to be fixed in
> > 1.10.1, please let me know. Thanks.
> >
> > Best Regards,
> > Yu
> >
>
Re: [DISCUSS] Releasing Flink 1.10.1
Posted by jincheng sun <su...@gmail.com>.
Thanks for bring up the discussion Yu. I would like to give you a hand at
the last stage when the RC is finished.(If you need) :)
Best,
Jincheng
Yu Li <ca...@gmail.com> 于2020年3月10日周二 下午5:49写道：
> Hi All,
>
> It has been almost one month since we released Flink 1.10.0. We already
> have more than 40 resolved improvements/bugs in the release-1.10 branch,
> and I propose to start the 1.10.1 release cycle.
>
> Most noticeable fixes are:
>
> - FLINK-16241 [legal] Remove the license and notice file in flink-ml-lib
> module
> - FLINK-16313 Fix RocksDB resource leak in flink-state-processor-api
> - FLINK-16161 Statistics zero should be known in HiveCatalog
> - FLINK-2336 ArrayIndexOufOBoundsException in TypeExtractor when mapping
> - FLINK-16108 StreamSQLExample is failed if running in blink planner
> - FLINK-16139 Co-location constraints are not reset on task recovery in
> DefaultScheduler
> - FLINK-16414 Create udaf/udtf function using sql casuing
> ValidationException: SQL validation failed
>
> Furthermore, I think the following issues should be merged before 1.10.1
> release (especially the Metaspace OOM issue):
>
> - FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job
> submission
> - FLINK-16406 Increase default value for JVM Metaspace to minimise its
> OutOfMemoryError
> - FLINK-16047 Blink planner produces wrong aggregate results with state
> clean up
> - FLINK-16070 Blink planner can not extract correct unique key for
> UpsertStreamTableSink
>
> I would volunteer as the release manager and kick off the release process
> once blocker issues are merged. What do you think?
>
> If there are any concerns or missing blocker issues need to be fixed in
> 1.10.1, please let me know. Thanks.
>
> Best Regards,
> Yu
>