You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by Ufuk Celebi <uc...@apache.org> on 2017/04/27 14:24:17 UTC

[DISCUSS] Feature Freeze

Hey devs! :-)

We decided to follow a time-based release model with the upcoming 1.3
release and the planned feature freeze is on Monday, May 1st.

I wanted to start a discussion to get a quick overview of the current
state of things.

- Is everyone on track and aware of the feature freeze? ;)
- Are there any major features we want in 1.3 that have not been merged yet?
- Do we need to extend the feature freeze, because of an important feature?

Would be great to gather a list of features/PRs that we want in the
1.3 release. This could be a good starting point for the release
manager (@Robert?).

Best,

Ufuk

Re: [DISCUSS] Feature Freeze

Posted by 施晓罡 <sh...@gmail.com>.

Hi Ufuk

Incremental checkpointing can significantly improve the performance of checkpointing.
I hope it can be included in release 1.3.

I am working with Stefan on the last subtask, which attempts to implement incremental checkpointing in RocksDB state backend (FLINK-6364). 
Without future optimizations, the work is near completion. So i think 2-3 more days would be good.

Regards
Xiaogang

> 在 2017年4月28日，上午3:22，Chesnay Schepler <ch...@apache.org> 写道：
> 
> Hello,
> 
> FLINK-5892 (Restoring state by operator) is also nearing completion, but with only 1 day left before the weekend we're cutting it really short.
> 
> Since this eliminates a major pain point when updating jobs, as it allows the modification of chains, another day or 2 would be good i think.
> 
> Regards,
> Chesnay
> 
> On 27.04.2017 18:55, Bowen Li wrote:
>> Hi Ufuk,
>>      I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter) into
>> release 1.3. It's in the final state of code review in
>> https://github.com/apache/flink/pull/3736
>> 
>> Thanks,
>> Bowen
>> 
>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>> wangzhijiang999@aliyun.com> wrote:
>> 
>>> Hi Ufuk,
>>> Thank you for launching this topic!
>>> I wish my latest refinement of buffer provider (https://issues.apache.org/
>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
>>> get benefit from it. And I think it can be completed with the help of your
>>> reviews this week.
>>> Cheers,Zhijiang------------------------------------------------------------------发件人：Ufuk
>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>>> 题：[DISCUSS] Feature Freeze
>>> Hey devs! :-)
>>> 
>>> We decided to follow a time-based release model with the upcoming 1.3
>>> release and the planned feature freeze is on Monday, May 1st.
>>> 
>>> I wanted to start a discussion to get a quick overview of the current
>>> state of things.
>>> 
>>> - Is everyone on track and aware of the feature freeze? ;)
>>> - Are there any major features we want in 1.3 that
>>> have not been merged yet?
>>> - Do we need to extend the feature freeze, because of an
>>> important feature?
>>> 
>>> Would be great to gather a list of features/PRs that we want in the
>>> 1.3 release. This could be a good starting point for the release
>>> manager (@Robert?).
>>> 
>>> Best,
>>> 
>>> Ufuk
>>> 
>

Re: [DISCUSS] Feature Freeze

Posted by Chesnay Schepler <ch...@apache.org>.

It's not really a release candidate in that sense, but a common test 
release that everyone should work against.

This gives us a consistent view about the state at commit X, as opposed 
to testing directly against the branch where it is likely everyone works 
on a different version.

On 08.05.2017 09:56, Kostas Kloudas wrote:
> Hi Robert,
>
> Thanks for starting the process!
>
> My only remark is that given that the master is unstable, does it make sense to create an RC0?
>
> Kostas
>
>> On May 8, 2017, at 8:52 AM, Robert Metzger <rm...@apache.org> wrote:
>>
>> Great!
>> It also looks like the other big features made it also into master this
>> weekend.
>>
>> I'll now create the feature branch and create the testing RC0.
>>
>> On Sat, May 6, 2017 at 12:04 PM, Fabian Hueske <fh...@gmail.com> wrote:
>>
>>> I merged the last to major features for the Table API / SQL (time
>>> indicators and retraction support) to master.
>>> We will need to work on some smaller issues for those features which will
>>> take a few more days (1 week max), but the big changes are in.
>>>
>>> Working on those final issue does not block a release candidate. The Table
>>> API / SQL are on top of the DataStream API and runtime.
>>> So the last fixes won't interfere with testing the lower levels of the
>>> system.
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-05 21:02 GMT+02:00 Stephan Ewen <se...@apache.org>:
>>>
>>>> Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.
>>>>
>>>> Thanks to the community for hard work on the features.
>>>>
>>>> On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>>>
>>>>> On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
>>>>>> Also, if no release candidate would be created today, it would not
>>> make
>>>>> any
>>>>>> difference anyways...
>>>>> If no one tests a RC (if created today) over the weekend it also
>>>>> wouldn't make a difference. ;-)
>>>>>
>>>>> Thanks to all for chiming in here and Robert and Aljoscha especially
>>>>> for making sure that everyone is on the same page wrt blockers etc.
>>>>>
>

Re: [DISCUSS] Feature Freeze

Posted by Robert Metzger <rm...@apache.org>.

It depends :)
If the unstable tests are caused by a bug in the tests itself, its not an
issue. If its a blocking issue in one of the core components, I'll
immediately create a new RC.
Part of the reason why I'm creating the RC0 is also to ensure that the
build infrastructure properly works.

On Mon, May 8, 2017 at 9:56 AM, Kostas Kloudas <k....@data-artisans.com>
wrote:

> Hi Robert,
>
> Thanks for starting the process!
>
> My only remark is that given that the master is unstable, does it make
> sense to create an RC0?
>
> Kostas
>
> > On May 8, 2017, at 8:52 AM, Robert Metzger <rm...@apache.org> wrote:
> >
> > Great!
> > It also looks like the other big features made it also into master this
> > weekend.
> >
> > I'll now create the feature branch and create the testing RC0.
> >
> > On Sat, May 6, 2017 at 12:04 PM, Fabian Hueske <fh...@gmail.com>
> wrote:
> >
> >> I merged the last to major features for the Table API / SQL (time
> >> indicators and retraction support) to master.
> >> We will need to work on some smaller issues for those features which
> will
> >> take a few more days (1 week max), but the big changes are in.
> >>
> >> Working on those final issue does not block a release candidate. The
> Table
> >> API / SQL are on top of the DataStream API and runtime.
> >> So the last fixes won't interfere with testing the lower levels of the
> >> system.
> >>
> >> Cheers, Fabian
> >>
> >> 2017-05-05 21:02 GMT+02:00 Stephan Ewen <se...@apache.org>:
> >>
> >>> Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.
> >>>
> >>> Thanks to the community for hard work on the features.
> >>>
> >>> On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >>>
> >>>> On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org>
> wrote:
> >>>>> Also, if no release candidate would be created today, it would not
> >> make
> >>>> any
> >>>>> difference anyways...
> >>>>
> >>>> If no one tests a RC (if created today) over the weekend it also
> >>>> wouldn't make a difference. ;-)
> >>>>
> >>>> Thanks to all for chiming in here and Robert and Aljoscha especially
> >>>> for making sure that everyone is on the same page wrt blockers etc.
> >>>>
> >>>
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Kostas Kloudas <k....@data-artisans.com>.

Hi Robert, 

Thanks for starting the process!

My only remark is that given that the master is unstable, does it make sense to create an RC0?

Kostas

> On May 8, 2017, at 8:52 AM, Robert Metzger <rm...@apache.org> wrote:
> 
> Great!
> It also looks like the other big features made it also into master this
> weekend.
> 
> I'll now create the feature branch and create the testing RC0.
> 
> On Sat, May 6, 2017 at 12:04 PM, Fabian Hueske <fh...@gmail.com> wrote:
> 
>> I merged the last to major features for the Table API / SQL (time
>> indicators and retraction support) to master.
>> We will need to work on some smaller issues for those features which will
>> take a few more days (1 week max), but the big changes are in.
>> 
>> Working on those final issue does not block a release candidate. The Table
>> API / SQL are on top of the DataStream API and runtime.
>> So the last fixes won't interfere with testing the lower levels of the
>> system.
>> 
>> Cheers, Fabian
>> 
>> 2017-05-05 21:02 GMT+02:00 Stephan Ewen <se...@apache.org>:
>> 
>>> Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.
>>> 
>>> Thanks to the community for hard work on the features.
>>> 
>>> On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>> 
>>>> On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
>>>>> Also, if no release candidate would be created today, it would not
>> make
>>>> any
>>>>> difference anyways...
>>>> 
>>>> If no one tests a RC (if created today) over the weekend it also
>>>> wouldn't make a difference. ;-)
>>>> 
>>>> Thanks to all for chiming in here and Robert and Aljoscha especially
>>>> for making sure that everyone is on the same page wrt blockers etc.
>>>> 
>>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Robert Metzger <rm...@apache.org>.

Great!
It also looks like the other big features made it also into master this
weekend.

I'll now create the feature branch and create the testing RC0.

On Sat, May 6, 2017 at 12:04 PM, Fabian Hueske <fh...@gmail.com> wrote:

> I merged the last to major features for the Table API / SQL (time
> indicators and retraction support) to master.
> We will need to work on some smaller issues for those features which will
> take a few more days (1 week max), but the big changes are in.
>
> Working on those final issue does not block a release candidate. The Table
> API / SQL are on top of the DataStream API and runtime.
> So the last fixes won't interfere with testing the lower levels of the
> system.
>
> Cheers, Fabian
>
> 2017-05-05 21:02 GMT+02:00 Stephan Ewen <se...@apache.org>:
>
> > Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.
> >
> > Thanks to the community for hard work on the features.
> >
> > On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >
> > > On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
> > > > Also, if no release candidate would be created today, it would not
> make
> > > any
> > > > difference anyways...
> > >
> > > If no one tests a RC (if created today) over the weekend it also
> > > wouldn't make a difference. ;-)
> > >
> > > Thanks to all for chiming in here and Robert and Aljoscha especially
> > > for making sure that everyone is on the same page wrt blockers etc.
> > >
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Fabian Hueske <fh...@gmail.com>.

I merged the last to major features for the Table API / SQL (time
indicators and retraction support) to master.
We will need to work on some smaller issues for those features which will
take a few more days (1 week max), but the big changes are in.

Working on those final issue does not block a release candidate. The Table
API / SQL are on top of the DataStream API and runtime.
So the last fixes won't interfere with testing the lower levels of the
system.

Cheers, Fabian

2017-05-05 21:02 GMT+02:00 Stephan Ewen <se...@apache.org>:

> Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.
>
> Thanks to the community for hard work on the features.
>
> On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:
>
> > On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
> > > Also, if no release candidate would be created today, it would not make
> > any
> > > difference anyways...
> >
> > If no one tests a RC (if created today) over the weekend it also
> > wouldn't make a difference. ;-)
> >
> > Thanks to all for chiming in here and Robert and Aljoscha especially
> > for making sure that everyone is on the same page wrt blockers etc.
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Stephan Ewen <se...@apache.org>.

Yes, I second Ufuk, thanks Robert and Aljoscha for the effort.

Thanks to the community for hard work on the features.

On Fri, May 5, 2017 at 4:03 PM, Ufuk Celebi <uc...@apache.org> wrote:

> On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
> > Also, if no release candidate would be created today, it would not make
> any
> > difference anyways...
>
> If no one tests a RC (if created today) over the weekend it also
> wouldn't make a difference. ;-)
>
> Thanks to all for chiming in here and Robert and Aljoscha especially
> for making sure that everyone is on the same page wrt blockers etc.
>

Re: [DISCUSS] Feature Freeze

Posted by Ufuk Celebi <uc...@apache.org>.

On Fri, May 5, 2017 at 3:59 PM, Stephan Ewen <se...@apache.org> wrote:
> Also, if no release candidate would be created today, it would not make any
> difference anyways...

If no one tests a RC (if created today) over the weekend it also
wouldn't make a difference. ;-)

Thanks to all for chiming in here and Robert and Aljoscha especially
for making sure that everyone is on the same page wrt blockers etc.

Re: [DISCUSS] Feature Freeze

Posted by Stephan Ewen <se...@apache.org>.

@robert - thanks, I think that is nice. That gives fairness across
timezones.
Also, if no release candidate would be created today, it would not make any
difference anyways...

On Fri, May 5, 2017 at 3:44 PM, Robert Metzger <rm...@apache.org> wrote:

> To quickly clarify: I said Friday 3pm for the feature freeze in my Tuesday
> email, but I decided to actually turn it into a proper *"end of the week"
> by doing the feature freeze Sunday evening/Monday morning*.
>
> On Fri, May 5, 2017 at 11:46 AM, Radu Tudoran <ra...@huawei.com>
> wrote:
>
> > Sorry in case I missed something in the discussion of coming up with the
> > list of open threads, but there is also
> >
> > [FLINK-6075] Support Order By for Stream SQL (Support Limit/Top(Sort) for
> > Stream SQL)
> >
> > ...in case you want to consider it (it only needs a review)
> >
> > Best regards,
> >
> > -----Original Message-----
> > From: Robert Metzger [mailto:rmetzger@apache.org]
> > Sent: Friday, May 05, 2017 10:43 AM
> > To: dev@flink.apache.org
> > Subject: Re: [DISCUSS] Feature Freeze
> >
> > I've checked the release status again, and it seems that some more issues
> > made it into master, while other very big new features are still being
> > finished (incremental checkpointing, retractions, serializer upgrades and
> > automatic buffer management).
> >
> > I'll publish the first, non-voting, testing only RC0 on Monday morning.
> >
> >
> > Completed Features for 1.3
> > - TODO after feature freeze
> >
> > Blockers:
> > 12 blockers (4 documentation, 1 refcatoring) --> 8 real blockers (can be
> > fixed after the feature freeze)
> >
> >
> > Really important (open):
> > - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
> > (pending PR)
> > - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
> > FLINK-6093)
> > - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
> > under review)
> > - FLINK-6178: Allow upgrades to state serializers (pending PR, no review
> )
> >
> >
> > Mentioned in the thread (open):
> > - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> > - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending
> PR)
> > - FLINK-6373: Add runtime support for distinct aggregation over grouped
> > windows (pending PR, under review)
> > - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> > - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
> > review)
> > - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
> > review)
> > - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> > (pending PR, no review)
> > - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
> > Connector (pending PR, under review)
> >
> > Closed:
> > - FLINK-6377: Support map types in the Table / SQL API (merged)
> > - FLINK-6398: RowSerializer's duplicate should always return a new
> instance
> > (merged)
> > - FLINK-5998: Un-fat Hadoop from Flink fat jar (merged)
> > - FLINK-6337: Remove the buffer provider from
> PartitionRequestServerHandler
> > (merged)
> > - FLINK-5892: Recover job state at the granularity of operator (merged)
> > - FLINK-5906: Add support to register user defined aggregates in
> > TableEnvironment (merged)
> > - FLINK-6334: Refactoring UDTF interface (merged)
> >
> >
> >
> > On Wed, May 3, 2017 at 11:42 AM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> > > This is the list of blocking issues for Flink 1.3:
> > > https://issues.apache.org/jira/browse/FLINK-5740?jql=project
> > > %20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%
> > > 20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%
> > > 20Blocker%20AND%20fixVersion%20%3D%201.3.0
> > > <https://issues.apache.org/jir
> > > a/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%
> > > 20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%
> > > 20Blocker%20AND%20fixVersion%20=%201.3.0>
> > >
> > > Could everyone please update the state of the issues mentioned in this
> > > thread to “blocking” and set the version to “1.3.0” if they feel that
> > > this is appropriate. Otherwise it is very hard for the release manager
> > > to get an overview over the progress of the release. With an
> > > up-to-date list it’s also possible to triage issues and discuss on the
> > > issue whether it really should be a release blocker.
> > >
> > > Best,
> > > Aljoscha
> > > > On 2. May 2017, at 22:29, Eron Wright <er...@gmail.com> wrote:
> > > >
> > > > Robert, I'd like to see FLINK-5974
> > > > <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS
> > > > support) added to the list of important issues.  A PR is ready.
> > > >
> > > > On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <
> > > k.kloudas@data-artisans.com>
> > > > wrote:
> > > >
> > > >> The only thing that I want to add in the features to be added for
> > > >> 1.3 is the NOT pattern for the CEP library.
> > > >>
> > > >> There is an open PR here: https://github.com/apache/flink/pull/3798
> > > >> < https://github.com/apache/flink/pull/3798>
> > > >> which is not far from getting in.
> > > >>
> > > >> Kostas
> > > >>
> > > >>> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org>
> > > wrote:
> > > >>>
> > > >>> Thanks a lot Ufuk for starting the discussion regarding the 1.3
> > > >>> feature freeze.
> > > >>>
> > > >>> I didn't feature freeze yesterday (Monday) because it was a public
> > > >> holiday
> > > >>> here in Germany.
> > > >>>
> > > >>> I haven't made up my mind whether to do the feature freeze today
> > > >>> or
> > > not.
> > > >>> Many important features seem to be close to completion.
> > > >>> I don't think we can pick features and wait with the release until
> > > >>> they
> > > >> are
> > > >>> finished. The only thing I can imagine we could do is extend the
> > > >> deadline a
> > > >>> little bit to give people more time to complete.
> > > >>> But I'm strictly against multiple extensions.
> > > >>>
> > > >>> To make the discussion a bit easier, I've collected all the JIRAs
> > > >> mentioned
> > > >>> in this thread. I decided to group them into "really important"
> > > >>> and "important". I hope nobody is offended by my selection and I'm
> > > >>> very
> > > open
> > > >> to
> > > >>> discuss it.
> > > >>> It doesn't really matter anyways because we will not block the
> > > >>> release
> > > on
> > > >>> the completion of certain features.
> > > >>>
> > > >>>
> > > >>> Completed Features for 1.3
> > > >>> - TODO!
> > > >>>
> > > >>> Blockers:
> > > >>> - TODO!
> > > >>>
> > > >>>
> > > >>> Really important (open):
> > > >>> - FLINK-6364: Implement incremental checkpointing in
> > > RocksDBStateBackend
> > > >>> (pending PR)
> > > >>> - FLINK-5906: Add support to register user defined aggregates in
> > > >>> TableEnvironment (no PR)
> > > >>> - FLINK-6047: Add support for Retraction in Table API / SQL
> > > >>> (depends on
> > > >>> FLINK-6093)
> > > >>> - FLINK-6093: Implement and turn on retraction for table sink
> > > >>> (pending
> > > >> PR)
> > > >>> - FLINK-6334: Refactoring UDTF interface (pending PR, under
> > > >>> review)
> > > >>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under
> > > review)
> > > >>> - FLINK-4545: Flink automatically manages TM network buffer
> > > >>> (pending
> > > PR,
> > > >>> under review)
> > > >>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no
> > > review
> > > >> )
> > > >>>
> > > >>>
> > > >>> Really important (closed):
> > > >>> - FLINK-5892: Recover job state at the granularity of operator
> > > >>> (merged)
> > > >>>
> > > >>>
> > > >>>
> > > >>> Important (open):
> > > >>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
> > > >> review)
> > > >>> - FLINK-6337: Remove the buffer provider from
> > > >> PartitionRequestServerHandler
> > > >>> (pending PR, under review)
> > > >>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> > > >>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL
> > > >>> (pending
> > > >> PR)
> > > >>> - FLINK-6373: Add runtime support for distinct aggregation over
> > > >>> grouped windows (pending PR, under review)
> > > >>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> > > >>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR,
> > > >>> under
> > > >>> review)
> > > >>> - FLINK-6196: Support dynamic schema in Table Function (pending
> > > >>> PR,
> > > under
> > > >>> review)
> > > >>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> > > >> (pending
> > > >>> PR, no review)
> > > >>> - FLINK-4821: Implement rescalable non-partitioned state for
> > > >>> Kinesis Connector (pending PR, under review) Important (closed):
> > > >>> - FLINK-6377: Support map types in the Table / SQL API (merged)
> > > >>> - FLINK-6398: RowSerializer's duplicate should always return a new
> > > >> instance
> > > >>> (merged)
> > > >>>
> > > >>>
> > > >>> I'll now collect the list of finished features in 1.3 and the
> > > >>> blockers
> > > >> and
> > > >>> update above list.
> > > >>>
> > > >>> Stephan's email just came in, and I like the idea of freezing the
> > > feature
> > > >>> set now but extending the deadline to Friday for the branching.
> > > >>> (Friday
> > > >> 3pm
> > > >>> CEST)
> > > >>>
> > > >>> Does everybody agree to that proposal?
> > > >>>
> > > >>>
> > > >>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org>
> > > wrote:
> > > >>>
> > > >>>> Thanks all for the lively discussion about the feature freeze and
> > > >>>> how
> > > to
> > > >>>> proceed.
> > > >>>> Because we committed to a time-based release schedule, we should
> > > >>>> not
> > > >> break
> > > >>>> the feature freeze too badly, or we would just disable the
> > > >>>> time-based-release-policy at the very first time it would trigger.
> > > >>>>
> > > >>>> Here are a few thoughts about what we can do:
> > > >>>>
> > > >>>> - First of all, please note that feature freeze does not mean bug
> > > >>>> fix freeze. Important bug fixes can and should go in as part of
> > > >>>> the 1.3.0 release testing period
> > > >>>>
> > > >>>> - We should probably not add brand new features to the list at
> > > >>>> this
> > > >> point
> > > >>>> (that would most likely break the release schedule completely)
> > > >>>>
> > > >>>> - I think there is a case to include some features which are
> > > >>>> partially (but not fully) in master already, or exist basically
> > > >>>> completely as
> > > Pull
> > > >>>> Requests at this time.
> > > >>>>
> > > >>>> How about we do *freeze the set of features* for the release
> > > >>>> today,
> > > but
> > > >>>> *leave
> > > >>>> a grace period to get the complete-but-not-merged features in
> > > >>>> until
> > > the
> > > >> end
> > > >>>> of this week*?
> > > >>>> That would mean forking the release-1.3 branch end of this week.
> > > >>>>
> > > >>>>
> > > >>>> We should try and agree on which features fall under that list,
> > > >>>> and
> > > make
> > > >>>> sure each one has a committer committed to merging them in the
> > > >>>> next
> > > >> days.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
> > > >> tzulitai@apache.org>
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Hi,
> > > >>>>>
> > > >>>>> I have 2 parts to discuss about delaying the feature freeze for
> > > >>>>> a
> > > bit:
> > > >>>>>
> > > >>>>> 1. Allow upgrades to state serializers (FLINK-6178) The feature
> > > >>>>> is split up into 2 PRs, one of which is already open and
> > > >> the
> > > >>>>> second one coming up very soon.
> > > >>>>> PR #1 reconfigurable TypeSerializers: https://
> > > >>>>> github.com/apache/flink/pull/3804 PR #2 (will open soon)
> > > >>>>> integrate serializer reconfiguration into
> > > state
> > > >>>>> restore flow (FLINK-6425)
> > > >>>>>
> > > >>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
> > > >>>>> Kafka - partition & topic regex discovery (PR -
> > > >>>> https://github.com/apache/
> > > >>>>> flink/pull/3746)
> > > >>>>> Kinesis - rescalable Kinesis consumer (PR -
> > > https://github.com/apache/
> > > >>>>> flink/pull/3001)
> > > >>>>>
> > > >>>>> All above mentioned features are not yet merged to master. I
> > > >>>>> think an additional few days to the end of this week for the
> > > >>>>> feature freeze
> > > >> should
> > > >>>>> make this possible.
> > > >>>>>
> > > >>>>> Cheers,
> > > >>>>> Gordon
> > > >>>>>
> > > >>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler
> > > >>>>> (chesnay@apache.org
> > > )
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>> FLINK-5892 has been merged.
> > > >>>>>
> > > >>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is
> > > >>>>> also
> > > >> still
> > > >>>>> open and could use a second pair of eyes.
> > > >>>>>
> > > >>>>> On 28.04.2017 17:03, Kurt Young wrote:
> > > >>>>>> Hi Flavio,
> > > >>>>>>
> > > >>>>>> I have also fix the issue in 1.2 branch, but the next release
> > > >>>>>> will
> > > be
> > > >>>>> 1.2.2
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Kurt
> > > >>>>>>
> > > >>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com>
> > > wrote:
> > > >>>>>>
> > > >>>>>>> Flavio:
> > > >>>>>>> Have you seen this (w.r.t. 1.2.1) ?
> > > >>>>>>>
> > > >>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> > > >>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> > > >>>>>>>
> > > >>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> > > >>>>> pompermaier@okkam.it>
> > > >>>>>>> wrote:
> > > >>>>>>>
> > > >>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign
> > > >>>>>>>> Flink
> > > >>>> 1.2.0
> > > >>>>>>> in
> > > >>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer
> > > >>>>>>>> option
> > > in
> > > >>>> the
> > > >>>>>>>> short term..
> > > >>>>>>>>
> > > >>>>>>>> Best,
> > > >>>>>>>> Flavio
> > > >>>>>>>>
> > > >>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> > > >>>> aljoscha@apache.org
> > > >>>>>>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Ah, I see. The fix for that has been merged into master so
> > > >>>>>>>>> it
> > > will
> > > >>>> be
> > > >>>>>>>>> release in Flink 1.3.
> > > >>>>>>>>>
> > > >>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> > > >>>> pompermaier@okkam.it>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>> Sorry, you're right Aljosha..the issue number is correct,
> > > >>>>>>>>>> the
> > > link
> > > >>>> is
> > > >>>>>>>>>> wrong! The correct one is https://issues.apache.org/
> > > >>>>>>>>> jira/browse/FLINK-6398
> > > >>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> > > >>>>>>>> aljoscha@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> I think there might be a typo. We haven’t yet reached
> > > >>>>>>>>>>> issue
> > > >> number
> > > >>>>>>>> 6389,
> > > >>>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is
> 6410.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> > > >>>>>>> pompermaier@okkam.it>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>> If it's not a problem it will be great for us to include
> > > >>>>>>>>>>>> also
> > > >>>>>>>>> FLINK-6398
> > > >>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if
> > > >>>>>>>>>>>> it's
> > > not
> > > >> a
> > > >>>>>>> big
> > > >>>>>>>>>>> deal
> > > >>>>>>>>>>>> Best,
> > > >>>>>>>>>>>> Flavio
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> > > >>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
> > > >>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hi Devs,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for the release plan.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Could you also please add the feature FLINK-6196
> > > >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196>
> > > >>>>>>>>>>>>> Support
> > > >>>>>>> dynamic
> > > >>>>>>>>>>> schema
> > > >>>>>>>>>>>>> in Table Function?
> > > >>>>>>>>>>>>> I’d like to update the code as comments left on PR today.
> > > >>>>>>>>>>>>> I will try to make sure the code is updated before the
> > > >>>>>>>>>>>>> Apr
> > > >> 30th.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Zhuoluo
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hello,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks for starting this thread. It would be great to
> > > >>>>>>>>>>>>> see the
> > > >>>>>>>>> following
> > > >>>>>>>>>>>>> features available in Flink 1.3:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> > > >>>>>>>>>>>>> * Various improvements on SQL over group windows:
> > > >>>>>>>>>>>>> FLINK-6335,
> > > >>>>>>>>> FLINK-6373
> > > >>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
> > > >> FLINK-6225
> > > >>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> All of them have gone through at least one round of
> > > >>>>>>>>>>>>> review so
> > > >>>> I'm
> > > >>>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in,
> > > >>>>>>>>>>>>> but
> > > it
> > > >>>>>>>> depends
> > > >>>>>>>>>>> on
> > > >>>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Regards,
> > > >>>>>>>>>>>>> Haohui
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> > > >>>>>>>> chesnay@apache.org
> > > >>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hello,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> > > >>>>>>> completion,
> > > >>>>>>>>> but
> > > >>>>>>>>>>>>> with only 1 day left before the weekend we're cutting it
> > > really
> > > >>>>>>>> short.
> > > >>>>>>>>>>>>> Since this eliminates a major pain point when updating
> > > >>>>>>>>>>>>> jobs,
> > > as
> > > >>>> it
> > > >>>>>>>>>>>>> allows the modification of chains, another day or 2
> > > >>>>>>>>>>>>> would be
> > > >>>> good
> > > >>>>>>> i
> > > >>>>>>>>>>> think.
> > > >>>>>>>>>>>>> Regards,
> > > >>>>>>>>>>>>> Chesnay
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hi Ufuk,
> > > >>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> > > >>>>>>> reporter)
> > > >>>>>>>>>>>>> into
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> release 1.3. It's in the final state of code review in
> > > >>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>> Bowen
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM,
> > > >>>>>>>>>>>>> Zhijiang(wangzhijiang999) < wangzhijiang999@aliyun.com>
> > wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Hi Ufuk,
> > > >>>>>>>>>>>>> Thank you for launching this topic!
> > > >>>>>>>>>>>>> I wish my latest refinement of buffer provider (
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> https://issues.apache.org/
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most
> > > >>>>>>>>>>>>> of the
> > > >>>>>>> jobs
> > > >>>>>>>>> can
> > > >>>>>>>>>>>>> get benefit from it. And I think it can be completed
> > > >>>>>>>>>>>>> with the
> > > >>>> help
> > > >>>>>>>> of
> > > >>>>>>>>>>>>> your
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> reviews this week.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Cheers,Zhijiang-----------------------------------------
> > > >>>>>>>>>>>>> ----
> > > >>>>>>>>>>>>> ---------------------发件人：Ufuk
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev
> > > >>>>>>>>>>>>> <
> > > >>>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger
> > > >>>>>>>>>>>>> <rmetzger@apache.org
> > > >主
> > > >>>>>>>>>>>>> 题：[DISCUSS] Feature Freeze Hey devs! :-)
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> We decided to follow a time-based release model with the
> > > >>>> upcoming
> > > >>>>>>>> 1.3
> > > >>>>>>>>>>>>> release and the planned feature freeze is on Monday, May
> > 1st.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I wanted to start a discussion to get a quick overview
> > > >>>>>>>>>>>>> of the
> > > >>>>>>>> current
> > > >>>>>>>>>>>>> state of things.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> - Is everyone on track and aware of the feature freeze?
> > > >>>>>>>>>>>>> ;)
> > > >>>>>>>>>>>>> - Are there any major features we want in 1.3 that have
> > > >>>>>>>>>>>>> not been merged yet?
> > > >>>>>>>>>>>>> - Do we need to extend the feature freeze, because of an
> > > >>>>>>>>>>>>> important feature?
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Would be great to gather a list of features/PRs that we
> > > >>>>>>>>>>>>> want
> > > in
> > > >>>>>>> the
> > > >>>>>>>>>>>>> 1.3 release. This could be a good starting point for the
> > > >> release
> > > >>>>>>>>>>>>> manager (@Robert?).
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Best,
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Ufuk
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > > >>
> > >
> > >
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Robert Metzger <rm...@apache.org>.

To quickly clarify: I said Friday 3pm for the feature freeze in my Tuesday
email, but I decided to actually turn it into a proper *"end of the week"
by doing the feature freeze Sunday evening/Monday morning*.

On Fri, May 5, 2017 at 11:46 AM, Radu Tudoran <ra...@huawei.com>
wrote:

> Sorry in case I missed something in the discussion of coming up with the
> list of open threads, but there is also
>
> [FLINK-6075] Support Order By for Stream SQL (Support Limit/Top(Sort) for
> Stream SQL)
>
> ...in case you want to consider it (it only needs a review)
>
> Best regards,
>
> -----Original Message-----
> From: Robert Metzger [mailto:rmetzger@apache.org]
> Sent: Friday, May 05, 2017 10:43 AM
> To: dev@flink.apache.org
> Subject: Re: [DISCUSS] Feature Freeze
>
> I've checked the release status again, and it seems that some more issues
> made it into master, while other very big new features are still being
> finished (incremental checkpointing, retractions, serializer upgrades and
> automatic buffer management).
>
> I'll publish the first, non-voting, testing only RC0 on Monday morning.
>
>
> Completed Features for 1.3
> - TODO after feature freeze
>
> Blockers:
> 12 blockers (4 documentation, 1 refcatoring) --> 8 real blockers (can be
> fixed after the feature freeze)
>
>
> Really important (open):
> - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
> (pending PR)
> - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
> FLINK-6093)
> - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
> under review)
> - FLINK-6178: Allow upgrades to state serializers (pending PR, no review )
>
>
> Mentioned in the thread (open):
> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending PR)
> - FLINK-6373: Add runtime support for distinct aggregation over grouped
> windows (pending PR, under review)
> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
> review)
> - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
> review)
> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> (pending PR, no review)
> - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
> Connector (pending PR, under review)
>
> Closed:
> - FLINK-6377: Support map types in the Table / SQL API (merged)
> - FLINK-6398: RowSerializer's duplicate should always return a new instance
> (merged)
> - FLINK-5998: Un-fat Hadoop from Flink fat jar (merged)
> - FLINK-6337: Remove the buffer provider from PartitionRequestServerHandler
> (merged)
> - FLINK-5892: Recover job state at the granularity of operator (merged)
> - FLINK-5906: Add support to register user defined aggregates in
> TableEnvironment (merged)
> - FLINK-6334: Refactoring UDTF interface (merged)
>
>
>
> On Wed, May 3, 2017 at 11:42 AM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > This is the list of blocking issues for Flink 1.3:
> > https://issues.apache.org/jira/browse/FLINK-5740?jql=project
> > %20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%
> > 20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%
> > 20Blocker%20AND%20fixVersion%20%3D%201.3.0
> > <https://issues.apache.org/jir
> > a/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%
> > 20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%
> > 20Blocker%20AND%20fixVersion%20=%201.3.0>
> >
> > Could everyone please update the state of the issues mentioned in this
> > thread to “blocking” and set the version to “1.3.0” if they feel that
> > this is appropriate. Otherwise it is very hard for the release manager
> > to get an overview over the progress of the release. With an
> > up-to-date list it’s also possible to triage issues and discuss on the
> > issue whether it really should be a release blocker.
> >
> > Best,
> > Aljoscha
> > > On 2. May 2017, at 22:29, Eron Wright <er...@gmail.com> wrote:
> > >
> > > Robert, I'd like to see FLINK-5974
> > > <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS
> > > support) added to the list of important issues.  A PR is ready.
> > >
> > > On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <
> > k.kloudas@data-artisans.com>
> > > wrote:
> > >
> > >> The only thing that I want to add in the features to be added for
> > >> 1.3 is the NOT pattern for the CEP library.
> > >>
> > >> There is an open PR here: https://github.com/apache/flink/pull/3798
> > >> < https://github.com/apache/flink/pull/3798>
> > >> which is not far from getting in.
> > >>
> > >> Kostas
> > >>
> > >>> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org>
> > wrote:
> > >>>
> > >>> Thanks a lot Ufuk for starting the discussion regarding the 1.3
> > >>> feature freeze.
> > >>>
> > >>> I didn't feature freeze yesterday (Monday) because it was a public
> > >> holiday
> > >>> here in Germany.
> > >>>
> > >>> I haven't made up my mind whether to do the feature freeze today
> > >>> or
> > not.
> > >>> Many important features seem to be close to completion.
> > >>> I don't think we can pick features and wait with the release until
> > >>> they
> > >> are
> > >>> finished. The only thing I can imagine we could do is extend the
> > >> deadline a
> > >>> little bit to give people more time to complete.
> > >>> But I'm strictly against multiple extensions.
> > >>>
> > >>> To make the discussion a bit easier, I've collected all the JIRAs
> > >> mentioned
> > >>> in this thread. I decided to group them into "really important"
> > >>> and "important". I hope nobody is offended by my selection and I'm
> > >>> very
> > open
> > >> to
> > >>> discuss it.
> > >>> It doesn't really matter anyways because we will not block the
> > >>> release
> > on
> > >>> the completion of certain features.
> > >>>
> > >>>
> > >>> Completed Features for 1.3
> > >>> - TODO!
> > >>>
> > >>> Blockers:
> > >>> - TODO!
> > >>>
> > >>>
> > >>> Really important (open):
> > >>> - FLINK-6364: Implement incremental checkpointing in
> > RocksDBStateBackend
> > >>> (pending PR)
> > >>> - FLINK-5906: Add support to register user defined aggregates in
> > >>> TableEnvironment (no PR)
> > >>> - FLINK-6047: Add support for Retraction in Table API / SQL
> > >>> (depends on
> > >>> FLINK-6093)
> > >>> - FLINK-6093: Implement and turn on retraction for table sink
> > >>> (pending
> > >> PR)
> > >>> - FLINK-6334: Refactoring UDTF interface (pending PR, under
> > >>> review)
> > >>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under
> > review)
> > >>> - FLINK-4545: Flink automatically manages TM network buffer
> > >>> (pending
> > PR,
> > >>> under review)
> > >>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no
> > review
> > >> )
> > >>>
> > >>>
> > >>> Really important (closed):
> > >>> - FLINK-5892: Recover job state at the granularity of operator
> > >>> (merged)
> > >>>
> > >>>
> > >>>
> > >>> Important (open):
> > >>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
> > >> review)
> > >>> - FLINK-6337: Remove the buffer provider from
> > >> PartitionRequestServerHandler
> > >>> (pending PR, under review)
> > >>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> > >>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL
> > >>> (pending
> > >> PR)
> > >>> - FLINK-6373: Add runtime support for distinct aggregation over
> > >>> grouped windows (pending PR, under review)
> > >>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> > >>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR,
> > >>> under
> > >>> review)
> > >>> - FLINK-6196: Support dynamic schema in Table Function (pending
> > >>> PR,
> > under
> > >>> review)
> > >>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> > >> (pending
> > >>> PR, no review)
> > >>> - FLINK-4821: Implement rescalable non-partitioned state for
> > >>> Kinesis Connector (pending PR, under review) Important (closed):
> > >>> - FLINK-6377: Support map types in the Table / SQL API (merged)
> > >>> - FLINK-6398: RowSerializer's duplicate should always return a new
> > >> instance
> > >>> (merged)
> > >>>
> > >>>
> > >>> I'll now collect the list of finished features in 1.3 and the
> > >>> blockers
> > >> and
> > >>> update above list.
> > >>>
> > >>> Stephan's email just came in, and I like the idea of freezing the
> > feature
> > >>> set now but extending the deadline to Friday for the branching.
> > >>> (Friday
> > >> 3pm
> > >>> CEST)
> > >>>
> > >>> Does everybody agree to that proposal?
> > >>>
> > >>>
> > >>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org>
> > wrote:
> > >>>
> > >>>> Thanks all for the lively discussion about the feature freeze and
> > >>>> how
> > to
> > >>>> proceed.
> > >>>> Because we committed to a time-based release schedule, we should
> > >>>> not
> > >> break
> > >>>> the feature freeze too badly, or we would just disable the
> > >>>> time-based-release-policy at the very first time it would trigger.
> > >>>>
> > >>>> Here are a few thoughts about what we can do:
> > >>>>
> > >>>> - First of all, please note that feature freeze does not mean bug
> > >>>> fix freeze. Important bug fixes can and should go in as part of
> > >>>> the 1.3.0 release testing period
> > >>>>
> > >>>> - We should probably not add brand new features to the list at
> > >>>> this
> > >> point
> > >>>> (that would most likely break the release schedule completely)
> > >>>>
> > >>>> - I think there is a case to include some features which are
> > >>>> partially (but not fully) in master already, or exist basically
> > >>>> completely as
> > Pull
> > >>>> Requests at this time.
> > >>>>
> > >>>> How about we do *freeze the set of features* for the release
> > >>>> today,
> > but
> > >>>> *leave
> > >>>> a grace period to get the complete-but-not-merged features in
> > >>>> until
> > the
> > >> end
> > >>>> of this week*?
> > >>>> That would mean forking the release-1.3 branch end of this week.
> > >>>>
> > >>>>
> > >>>> We should try and agree on which features fall under that list,
> > >>>> and
> > make
> > >>>> sure each one has a committer committed to merging them in the
> > >>>> next
> > >> days.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
> > >> tzulitai@apache.org>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi,
> > >>>>>
> > >>>>> I have 2 parts to discuss about delaying the feature freeze for
> > >>>>> a
> > bit:
> > >>>>>
> > >>>>> 1. Allow upgrades to state serializers (FLINK-6178) The feature
> > >>>>> is split up into 2 PRs, one of which is already open and
> > >> the
> > >>>>> second one coming up very soon.
> > >>>>> PR #1 reconfigurable TypeSerializers: https://
> > >>>>> github.com/apache/flink/pull/3804 PR #2 (will open soon)
> > >>>>> integrate serializer reconfiguration into
> > state
> > >>>>> restore flow (FLINK-6425)
> > >>>>>
> > >>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
> > >>>>> Kafka - partition & topic regex discovery (PR -
> > >>>> https://github.com/apache/
> > >>>>> flink/pull/3746)
> > >>>>> Kinesis - rescalable Kinesis consumer (PR -
> > https://github.com/apache/
> > >>>>> flink/pull/3001)
> > >>>>>
> > >>>>> All above mentioned features are not yet merged to master. I
> > >>>>> think an additional few days to the end of this week for the
> > >>>>> feature freeze
> > >> should
> > >>>>> make this possible.
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Gordon
> > >>>>>
> > >>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler
> > >>>>> (chesnay@apache.org
> > )
> > >>>>> wrote:
> > >>>>>
> > >>>>> FLINK-5892 has been merged.
> > >>>>>
> > >>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is
> > >>>>> also
> > >> still
> > >>>>> open and could use a second pair of eyes.
> > >>>>>
> > >>>>> On 28.04.2017 17:03, Kurt Young wrote:
> > >>>>>> Hi Flavio,
> > >>>>>>
> > >>>>>> I have also fix the issue in 1.2 branch, but the next release
> > >>>>>> will
> > be
> > >>>>> 1.2.2
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Kurt
> > >>>>>>
> > >>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com>
> > wrote:
> > >>>>>>
> > >>>>>>> Flavio:
> > >>>>>>> Have you seen this (w.r.t. 1.2.1) ?
> > >>>>>>>
> > >>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> > >>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> > >>>>>>>
> > >>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> > >>>>> pompermaier@okkam.it>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign
> > >>>>>>>> Flink
> > >>>> 1.2.0
> > >>>>>>> in
> > >>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer
> > >>>>>>>> option
> > in
> > >>>> the
> > >>>>>>>> short term..
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>> Flavio
> > >>>>>>>>
> > >>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> > >>>> aljoscha@apache.org
> > >>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Ah, I see. The fix for that has been merged into master so
> > >>>>>>>>> it
> > will
> > >>>> be
> > >>>>>>>>> release in Flink 1.3.
> > >>>>>>>>>
> > >>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> > >>>> pompermaier@okkam.it>
> > >>>>>>>>> wrote:
> > >>>>>>>>>> Sorry, you're right Aljosha..the issue number is correct,
> > >>>>>>>>>> the
> > link
> > >>>> is
> > >>>>>>>>>> wrong! The correct one is https://issues.apache.org/
> > >>>>>>>>> jira/browse/FLINK-6398
> > >>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> > >>>>>>>> aljoscha@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> I think there might be a typo. We haven’t yet reached
> > >>>>>>>>>>> issue
> > >> number
> > >>>>>>>> 6389,
> > >>>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> > >>>>>>> pompermaier@okkam.it>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>> If it's not a problem it will be great for us to include
> > >>>>>>>>>>>> also
> > >>>>>>>>> FLINK-6398
> > >>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if
> > >>>>>>>>>>>> it's
> > not
> > >> a
> > >>>>>>> big
> > >>>>>>>>>>> deal
> > >>>>>>>>>>>> Best,
> > >>>>>>>>>>>> Flavio
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> > >>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Devs,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for the release plan.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Could you also please add the feature FLINK-6196
> > >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196>
> > >>>>>>>>>>>>> Support
> > >>>>>>> dynamic
> > >>>>>>>>>>> schema
> > >>>>>>>>>>>>> in Table Function?
> > >>>>>>>>>>>>> I’d like to update the code as comments left on PR today.
> > >>>>>>>>>>>>> I will try to make sure the code is updated before the
> > >>>>>>>>>>>>> Apr
> > >> 30th.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Zhuoluo
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hello,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for starting this thread. It would be great to
> > >>>>>>>>>>>>> see the
> > >>>>>>>>> following
> > >>>>>>>>>>>>> features available in Flink 1.3:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> > >>>>>>>>>>>>> * Various improvements on SQL over group windows:
> > >>>>>>>>>>>>> FLINK-6335,
> > >>>>>>>>> FLINK-6373
> > >>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
> > >> FLINK-6225
> > >>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> All of them have gone through at least one round of
> > >>>>>>>>>>>>> review so
> > >>>> I'm
> > >>>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in,
> > >>>>>>>>>>>>> but
> > it
> > >>>>>>>> depends
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Regards,
> > >>>>>>>>>>>>> Haohui
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> > >>>>>>>> chesnay@apache.org
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hello,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> > >>>>>>> completion,
> > >>>>>>>>> but
> > >>>>>>>>>>>>> with only 1 day left before the weekend we're cutting it
> > really
> > >>>>>>>> short.
> > >>>>>>>>>>>>> Since this eliminates a major pain point when updating
> > >>>>>>>>>>>>> jobs,
> > as
> > >>>> it
> > >>>>>>>>>>>>> allows the modification of chains, another day or 2
> > >>>>>>>>>>>>> would be
> > >>>> good
> > >>>>>>> i
> > >>>>>>>>>>> think.
> > >>>>>>>>>>>>> Regards,
> > >>>>>>>>>>>>> Chesnay
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Ufuk,
> > >>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> > >>>>>>> reporter)
> > >>>>>>>>>>>>> into
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> release 1.3. It's in the final state of code review in
> > >>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>> Bowen
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM,
> > >>>>>>>>>>>>> Zhijiang(wangzhijiang999) < wangzhijiang999@aliyun.com>
> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Ufuk,
> > >>>>>>>>>>>>> Thank you for launching this topic!
> > >>>>>>>>>>>>> I wish my latest refinement of buffer provider (
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> https://issues.apache.org/
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most
> > >>>>>>>>>>>>> of the
> > >>>>>>> jobs
> > >>>>>>>>> can
> > >>>>>>>>>>>>> get benefit from it. And I think it can be completed
> > >>>>>>>>>>>>> with the
> > >>>> help
> > >>>>>>>> of
> > >>>>>>>>>>>>> your
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> reviews this week.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cheers,Zhijiang-----------------------------------------
> > >>>>>>>>>>>>> ----
> > >>>>>>>>>>>>> ---------------------发件人：Ufuk
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev
> > >>>>>>>>>>>>> <
> > >>>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger
> > >>>>>>>>>>>>> <rmetzger@apache.org
> > >主
> > >>>>>>>>>>>>> 题：[DISCUSS] Feature Freeze Hey devs! :-)
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We decided to follow a time-based release model with the
> > >>>> upcoming
> > >>>>>>>> 1.3
> > >>>>>>>>>>>>> release and the planned feature freeze is on Monday, May
> 1st.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I wanted to start a discussion to get a quick overview
> > >>>>>>>>>>>>> of the
> > >>>>>>>> current
> > >>>>>>>>>>>>> state of things.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> - Is everyone on track and aware of the feature freeze?
> > >>>>>>>>>>>>> ;)
> > >>>>>>>>>>>>> - Are there any major features we want in 1.3 that have
> > >>>>>>>>>>>>> not been merged yet?
> > >>>>>>>>>>>>> - Do we need to extend the feature freeze, because of an
> > >>>>>>>>>>>>> important feature?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Would be great to gather a list of features/PRs that we
> > >>>>>>>>>>>>> want
> > in
> > >>>>>>> the
> > >>>>>>>>>>>>> 1.3 release. This could be a good starting point for the
> > >> release
> > >>>>>>>>>>>>> manager (@Robert?).
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Best,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Ufuk
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>

RE: [DISCUSS] Feature Freeze

Posted by Radu Tudoran <ra...@huawei.com>.

Sorry in case I missed something in the discussion of coming up with the list of open threads, but there is also 

[FLINK-6075] Support Order By for Stream SQL (Support Limit/Top(Sort) for Stream SQL)

...in case you want to consider it (it only needs a review)

Best regards,

-----Original Message-----
From: Robert Metzger [mailto:rmetzger@apache.org] 
Sent: Friday, May 05, 2017 10:43 AM
To: dev@flink.apache.org
Subject: Re: [DISCUSS] Feature Freeze

I've checked the release status again, and it seems that some more issues made it into master, while other very big new features are still being finished (incremental checkpointing, retractions, serializer upgrades and automatic buffer management).

I'll publish the first, non-voting, testing only RC0 on Monday morning.


Completed Features for 1.3
- TODO after feature freeze

Blockers:
12 blockers (4 documentation, 1 refcatoring) --> 8 real blockers (can be fixed after the feature freeze)


Really important (open):
- FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend (pending PR)
- FLINK-6047: Add support for Retraction in Table API / SQL (depends on
FLINK-6093)
- FLINK-4545: Flink automatically manages TM network buffer (pending PR, under review)
- FLINK-6178: Allow upgrades to state serializers (pending PR, no review )


Mentioned in the thread (open):
- FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
- FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending PR)
- FLINK-6373: Add runtime support for distinct aggregation over grouped windows (pending PR, under review)
- FLINK-6281: Create TableSink for JDBC (pending PR, under review)
- FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
review)
- FLINK-6196: Support dynamic schema in Table Function (pending PR, under
review)
- FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer (pending PR, no review)
- FLINK-4821: Implement rescalable non-partitioned state for Kinesis Connector (pending PR, under review)

Closed:
- FLINK-6377: Support map types in the Table / SQL API (merged)
- FLINK-6398: RowSerializer's duplicate should always return a new instance
(merged)
- FLINK-5998: Un-fat Hadoop from Flink fat jar (merged)
- FLINK-6337: Remove the buffer provider from PartitionRequestServerHandler
(merged)
- FLINK-5892: Recover job state at the granularity of operator (merged)
- FLINK-5906: Add support to register user defined aggregates in TableEnvironment (merged)
- FLINK-6334: Refactoring UDTF interface (merged)



On Wed, May 3, 2017 at 11:42 AM, Aljoscha Krettek <al...@apache.org>
wrote:

> This is the list of blocking issues for Flink 1.3:
> https://issues.apache.org/jira/browse/FLINK-5740?jql=project
> %20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%
> 20Blocker%20AND%20fixVersion%20%3D%201.3.0 
> <https://issues.apache.org/jir 
> a/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%
> 20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%
> 20Blocker%20AND%20fixVersion%20=%201.3.0>
>
> Could everyone please update the state of the issues mentioned in this 
> thread to “blocking” and set the version to “1.3.0” if they feel that 
> this is appropriate. Otherwise it is very hard for the release manager 
> to get an overview over the progress of the release. With an 
> up-to-date list it’s also possible to triage issues and discuss on the 
> issue whether it really should be a release blocker.
>
> Best,
> Aljoscha
> > On 2. May 2017, at 22:29, Eron Wright <er...@gmail.com> wrote:
> >
> > Robert, I'd like to see FLINK-5974
> > <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS 
> > support) added to the list of important issues.  A PR is ready.
> >
> > On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <
> k.kloudas@data-artisans.com>
> > wrote:
> >
> >> The only thing that I want to add in the features to be added for 
> >> 1.3 is the NOT pattern for the CEP library.
> >>
> >> There is an open PR here: https://github.com/apache/flink/pull/3798 
> >> < https://github.com/apache/flink/pull/3798>
> >> which is not far from getting in.
> >>
> >> Kostas
> >>
> >>> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org>
> wrote:
> >>>
> >>> Thanks a lot Ufuk for starting the discussion regarding the 1.3 
> >>> feature freeze.
> >>>
> >>> I didn't feature freeze yesterday (Monday) because it was a public
> >> holiday
> >>> here in Germany.
> >>>
> >>> I haven't made up my mind whether to do the feature freeze today 
> >>> or
> not.
> >>> Many important features seem to be close to completion.
> >>> I don't think we can pick features and wait with the release until 
> >>> they
> >> are
> >>> finished. The only thing I can imagine we could do is extend the
> >> deadline a
> >>> little bit to give people more time to complete.
> >>> But I'm strictly against multiple extensions.
> >>>
> >>> To make the discussion a bit easier, I've collected all the JIRAs
> >> mentioned
> >>> in this thread. I decided to group them into "really important" 
> >>> and "important". I hope nobody is offended by my selection and I'm 
> >>> very
> open
> >> to
> >>> discuss it.
> >>> It doesn't really matter anyways because we will not block the 
> >>> release
> on
> >>> the completion of certain features.
> >>>
> >>>
> >>> Completed Features for 1.3
> >>> - TODO!
> >>>
> >>> Blockers:
> >>> - TODO!
> >>>
> >>>
> >>> Really important (open):
> >>> - FLINK-6364: Implement incremental checkpointing in
> RocksDBStateBackend
> >>> (pending PR)
> >>> - FLINK-5906: Add support to register user defined aggregates in 
> >>> TableEnvironment (no PR)
> >>> - FLINK-6047: Add support for Retraction in Table API / SQL 
> >>> (depends on
> >>> FLINK-6093)
> >>> - FLINK-6093: Implement and turn on retraction for table sink 
> >>> (pending
> >> PR)
> >>> - FLINK-6334: Refactoring UDTF interface (pending PR, under 
> >>> review)
> >>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under
> review)
> >>> - FLINK-4545: Flink automatically manages TM network buffer 
> >>> (pending
> PR,
> >>> under review)
> >>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no
> review
> >> )
> >>>
> >>>
> >>> Really important (closed):
> >>> - FLINK-5892: Recover job state at the granularity of operator 
> >>> (merged)
> >>>
> >>>
> >>>
> >>> Important (open):
> >>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
> >> review)
> >>> - FLINK-6337: Remove the buffer provider from
> >> PartitionRequestServerHandler
> >>> (pending PR, under review)
> >>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> >>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL 
> >>> (pending
> >> PR)
> >>> - FLINK-6373: Add runtime support for distinct aggregation over 
> >>> grouped windows (pending PR, under review)
> >>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> >>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, 
> >>> under
> >>> review)
> >>> - FLINK-6196: Support dynamic schema in Table Function (pending 
> >>> PR,
> under
> >>> review)
> >>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> >> (pending
> >>> PR, no review)
> >>> - FLINK-4821: Implement rescalable non-partitioned state for 
> >>> Kinesis Connector (pending PR, under review) Important (closed):
> >>> - FLINK-6377: Support map types in the Table / SQL API (merged)
> >>> - FLINK-6398: RowSerializer's duplicate should always return a new
> >> instance
> >>> (merged)
> >>>
> >>>
> >>> I'll now collect the list of finished features in 1.3 and the 
> >>> blockers
> >> and
> >>> update above list.
> >>>
> >>> Stephan's email just came in, and I like the idea of freezing the
> feature
> >>> set now but extending the deadline to Friday for the branching. 
> >>> (Friday
> >> 3pm
> >>> CEST)
> >>>
> >>> Does everybody agree to that proposal?
> >>>
> >>>
> >>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org>
> wrote:
> >>>
> >>>> Thanks all for the lively discussion about the feature freeze and 
> >>>> how
> to
> >>>> proceed.
> >>>> Because we committed to a time-based release schedule, we should 
> >>>> not
> >> break
> >>>> the feature freeze too badly, or we would just disable the 
> >>>> time-based-release-policy at the very first time it would trigger.
> >>>>
> >>>> Here are a few thoughts about what we can do:
> >>>>
> >>>> - First of all, please note that feature freeze does not mean bug 
> >>>> fix freeze. Important bug fixes can and should go in as part of 
> >>>> the 1.3.0 release testing period
> >>>>
> >>>> - We should probably not add brand new features to the list at 
> >>>> this
> >> point
> >>>> (that would most likely break the release schedule completely)
> >>>>
> >>>> - I think there is a case to include some features which are 
> >>>> partially (but not fully) in master already, or exist basically 
> >>>> completely as
> Pull
> >>>> Requests at this time.
> >>>>
> >>>> How about we do *freeze the set of features* for the release 
> >>>> today,
> but
> >>>> *leave
> >>>> a grace period to get the complete-but-not-merged features in 
> >>>> until
> the
> >> end
> >>>> of this week*?
> >>>> That would mean forking the release-1.3 branch end of this week.
> >>>>
> >>>>
> >>>> We should try and agree on which features fall under that list, 
> >>>> and
> make
> >>>> sure each one has a committer committed to merging them in the 
> >>>> next
> >> days.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
> >> tzulitai@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> I have 2 parts to discuss about delaying the feature freeze for 
> >>>>> a
> bit:
> >>>>>
> >>>>> 1. Allow upgrades to state serializers (FLINK-6178) The feature 
> >>>>> is split up into 2 PRs, one of which is already open and
> >> the
> >>>>> second one coming up very soon.
> >>>>> PR #1 reconfigurable TypeSerializers: https://
> >>>>> github.com/apache/flink/pull/3804 PR #2 (will open soon) 
> >>>>> integrate serializer reconfiguration into
> state
> >>>>> restore flow (FLINK-6425)
> >>>>>
> >>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
> >>>>> Kafka - partition & topic regex discovery (PR -
> >>>> https://github.com/apache/
> >>>>> flink/pull/3746)
> >>>>> Kinesis - rescalable Kinesis consumer (PR -
> https://github.com/apache/
> >>>>> flink/pull/3001)
> >>>>>
> >>>>> All above mentioned features are not yet merged to master. I 
> >>>>> think an additional few days to the end of this week for the 
> >>>>> feature freeze
> >> should
> >>>>> make this possible.
> >>>>>
> >>>>> Cheers,
> >>>>> Gordon
> >>>>>
> >>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler 
> >>>>> (chesnay@apache.org
> )
> >>>>> wrote:
> >>>>>
> >>>>> FLINK-5892 has been merged.
> >>>>>
> >>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is 
> >>>>> also
> >> still
> >>>>> open and could use a second pair of eyes.
> >>>>>
> >>>>> On 28.04.2017 17:03, Kurt Young wrote:
> >>>>>> Hi Flavio,
> >>>>>>
> >>>>>> I have also fix the issue in 1.2 branch, but the next release 
> >>>>>> will
> be
> >>>>> 1.2.2
> >>>>>>
> >>>>>> Best,
> >>>>>> Kurt
> >>>>>>
> >>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >>>>>>
> >>>>>>> Flavio:
> >>>>>>> Have you seen this (w.r.t. 1.2.1) ?
> >>>>>>>
> >>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> >>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> >>>>>>>
> >>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> >>>>> pompermaier@okkam.it>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign 
> >>>>>>>> Flink
> >>>> 1.2.0
> >>>>>>> in
> >>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer 
> >>>>>>>> option
> in
> >>>> the
> >>>>>>>> short term..
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Flavio
> >>>>>>>>
> >>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> >>>> aljoscha@apache.org
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Ah, I see. The fix for that has been merged into master so 
> >>>>>>>>> it
> will
> >>>> be
> >>>>>>>>> release in Flink 1.3.
> >>>>>>>>>
> >>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> >>>> pompermaier@okkam.it>
> >>>>>>>>> wrote:
> >>>>>>>>>> Sorry, you're right Aljosha..the issue number is correct, 
> >>>>>>>>>> the
> link
> >>>> is
> >>>>>>>>>> wrong! The correct one is https://issues.apache.org/
> >>>>>>>>> jira/browse/FLINK-6398
> >>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> >>>>>>>> aljoscha@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I think there might be a typo. We haven’t yet reached 
> >>>>>>>>>>> issue
> >> number
> >>>>>>>> 6389,
> >>>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> >>>>>>>>>>>
> >>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> >>>>>>> pompermaier@okkam.it>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>> If it's not a problem it will be great for us to include 
> >>>>>>>>>>>> also
> >>>>>>>>> FLINK-6398
> >>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if 
> >>>>>>>>>>>> it's
> not
> >> a
> >>>>>>> big
> >>>>>>>>>>> deal
> >>>>>>>>>>>> Best,
> >>>>>>>>>>>> Flavio
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> >>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Devs,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the release plan.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Could you also please add the feature FLINK-6196 
> >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> 
> >>>>>>>>>>>>> Support
> >>>>>>> dynamic
> >>>>>>>>>>> schema
> >>>>>>>>>>>>> in Table Function?
> >>>>>>>>>>>>> I’d like to update the code as comments left on PR today.
> >>>>>>>>>>>>> I will try to make sure the code is updated before the 
> >>>>>>>>>>>>> Apr
> >> 30th.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Zhuoluo 
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for starting this thread. It would be great to 
> >>>>>>>>>>>>> see the
> >>>>>>>>> following
> >>>>>>>>>>>>> features available in Flink 1.3:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> >>>>>>>>>>>>> * Various improvements on SQL over group windows: 
> >>>>>>>>>>>>> FLINK-6335,
> >>>>>>>>> FLINK-6373
> >>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
> >> FLINK-6225
> >>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> All of them have gone through at least one round of 
> >>>>>>>>>>>>> review so
> >>>> I'm
> >>>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in, 
> >>>>>>>>>>>>> but
> it
> >>>>>>>> depends
> >>>>>>>>>>> on
> >>>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Haohui
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> >>>>>>>> chesnay@apache.org
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> >>>>>>> completion,
> >>>>>>>>> but
> >>>>>>>>>>>>> with only 1 day left before the weekend we're cutting it
> really
> >>>>>>>> short.
> >>>>>>>>>>>>> Since this eliminates a major pain point when updating 
> >>>>>>>>>>>>> jobs,
> as
> >>>> it
> >>>>>>>>>>>>> allows the modification of chains, another day or 2 
> >>>>>>>>>>>>> would be
> >>>> good
> >>>>>>> i
> >>>>>>>>>>> think.
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Chesnay
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> >>>>>>> reporter)
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> release 1.3. It's in the final state of code review in
> >>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Bowen
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, 
> >>>>>>>>>>>>> Zhijiang(wangzhijiang999) < wangzhijiang999@aliyun.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>>>> Thank you for launching this topic!
> >>>>>>>>>>>>> I wish my latest refinement of buffer provider (
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most 
> >>>>>>>>>>>>> of the
> >>>>>>> jobs
> >>>>>>>>> can
> >>>>>>>>>>>>> get benefit from it. And I think it can be completed 
> >>>>>>>>>>>>> with the
> >>>> help
> >>>>>>>> of
> >>>>>>>>>>>>> your
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> reviews this week.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,Zhijiang-----------------------------------------
> >>>>>>>>>>>>> ----
> >>>>>>>>>>>>> ---------------------发件人：Ufuk
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev 
> >>>>>>>>>>>>> <
> >>>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger 
> >>>>>>>>>>>>> <rmetzger@apache.org
> >主
> >>>>>>>>>>>>> 题：[DISCUSS] Feature Freeze Hey devs! :-)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We decided to follow a time-based release model with the
> >>>> upcoming
> >>>>>>>> 1.3
> >>>>>>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wanted to start a discussion to get a quick overview 
> >>>>>>>>>>>>> of the
> >>>>>>>> current
> >>>>>>>>>>>>> state of things.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Is everyone on track and aware of the feature freeze? 
> >>>>>>>>>>>>> ;)
> >>>>>>>>>>>>> - Are there any major features we want in 1.3 that have 
> >>>>>>>>>>>>> not been merged yet?
> >>>>>>>>>>>>> - Do we need to extend the feature freeze, because of an 
> >>>>>>>>>>>>> important feature?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Would be great to gather a list of features/PRs that we 
> >>>>>>>>>>>>> want
> in
> >>>>>>> the
> >>>>>>>>>>>>> 1.3 release. This could be a good starting point for the
> >> release
> >>>>>>>>>>>>> manager (@Robert?).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ufuk
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Robert Metzger <rm...@apache.org>.

I've checked the release status again, and it seems that some more issues
made it into master, while other very big new features are still being
finished (incremental checkpointing, retractions, serializer upgrades and
automatic buffer management).

I'll publish the first, non-voting, testing only RC0 on Monday morning.


Completed Features for 1.3
- TODO after feature freeze

Blockers:
12 blockers (4 documentation, 1 refcatoring) --> 8 real blockers (can be
fixed after the feature freeze)


Really important (open):
- FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
(pending PR)
- FLINK-6047: Add support for Retraction in Table API / SQL (depends on
FLINK-6093)
- FLINK-4545: Flink automatically manages TM network buffer (pending PR,
under review)
- FLINK-6178: Allow upgrades to state serializers (pending PR, no review )


Mentioned in the thread (open):
- FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
- FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending PR)
- FLINK-6373: Add runtime support for distinct aggregation over grouped
windows (pending PR, under review)
- FLINK-6281: Create TableSink for JDBC (pending PR, under review)
- FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
review)
- FLINK-6196: Support dynamic schema in Table Function (pending PR, under
review)
- FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer (pending
PR, no review)
- FLINK-4821: Implement rescalable non-partitioned state for Kinesis
Connector (pending PR, under review)

Closed:
- FLINK-6377: Support map types in the Table / SQL API (merged)
- FLINK-6398: RowSerializer's duplicate should always return a new instance
(merged)
- FLINK-5998: Un-fat Hadoop from Flink fat jar (merged)
- FLINK-6337: Remove the buffer provider from PartitionRequestServerHandler
(merged)
- FLINK-5892: Recover job state at the granularity of operator (merged)
- FLINK-5906: Add support to register user defined aggregates in
TableEnvironment (merged)
- FLINK-6334: Refactoring UDTF interface (merged)



On Wed, May 3, 2017 at 11:42 AM, Aljoscha Krettek <al...@apache.org>
wrote:

> This is the list of blocking issues for Flink 1.3:
> https://issues.apache.org/jira/browse/FLINK-5740?jql=project
> %20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%
> 20Blocker%20AND%20fixVersion%20%3D%201.3.0 <https://issues.apache.org/jir
> a/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%
> 20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%
> 20Blocker%20AND%20fixVersion%20=%201.3.0>
>
> Could everyone please update the state of the issues mentioned in this
> thread to “blocking” and set the version to “1.3.0” if they feel that this
> is appropriate. Otherwise it is very hard for the release manager to get an
> overview over the progress of the release. With an up-to-date list it’s
> also possible to triage issues and discuss on the issue whether it really
> should be a release blocker.
>
> Best,
> Aljoscha
> > On 2. May 2017, at 22:29, Eron Wright <er...@gmail.com> wrote:
> >
> > Robert, I'd like to see FLINK-5974
> > <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS support)
> > added to the list of important issues.  A PR is ready.
> >
> > On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <
> k.kloudas@data-artisans.com>
> > wrote:
> >
> >> The only thing that I want to add in the features to be added for 1.3
> >> is the NOT pattern for the CEP library.
> >>
> >> There is an open PR here: https://github.com/apache/flink/pull/3798 <
> >> https://github.com/apache/flink/pull/3798>
> >> which is not far from getting in.
> >>
> >> Kostas
> >>
> >>> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org>
> wrote:
> >>>
> >>> Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
> >>> freeze.
> >>>
> >>> I didn't feature freeze yesterday (Monday) because it was a public
> >> holiday
> >>> here in Germany.
> >>>
> >>> I haven't made up my mind whether to do the feature freeze today or
> not.
> >>> Many important features seem to be close to completion.
> >>> I don't think we can pick features and wait with the release until they
> >> are
> >>> finished. The only thing I can imagine we could do is extend the
> >> deadline a
> >>> little bit to give people more time to complete.
> >>> But I'm strictly against multiple extensions.
> >>>
> >>> To make the discussion a bit easier, I've collected all the JIRAs
> >> mentioned
> >>> in this thread. I decided to group them into "really important" and
> >>> "important". I hope nobody is offended by my selection and I'm very
> open
> >> to
> >>> discuss it.
> >>> It doesn't really matter anyways because we will not block the release
> on
> >>> the completion of certain features.
> >>>
> >>>
> >>> Completed Features for 1.3
> >>> - TODO!
> >>>
> >>> Blockers:
> >>> - TODO!
> >>>
> >>>
> >>> Really important (open):
> >>> - FLINK-6364: Implement incremental checkpointing in
> RocksDBStateBackend
> >>> (pending PR)
> >>> - FLINK-5906: Add support to register user defined aggregates in
> >>> TableEnvironment (no PR)
> >>> - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
> >>> FLINK-6093)
> >>> - FLINK-6093: Implement and turn on retraction for table sink (pending
> >> PR)
> >>> - FLINK-6334: Refactoring UDTF interface (pending PR, under review)
> >>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under
> review)
> >>> - FLINK-4545: Flink automatically manages TM network buffer (pending
> PR,
> >>> under review)
> >>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no
> review
> >> )
> >>>
> >>>
> >>> Really important (closed):
> >>> - FLINK-5892: Recover job state at the granularity of operator (merged)
> >>>
> >>>
> >>>
> >>> Important (open):
> >>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
> >> review)
> >>> - FLINK-6337: Remove the buffer provider from
> >> PartitionRequestServerHandler
> >>> (pending PR, under review)
> >>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> >>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending
> >> PR)
> >>> - FLINK-6373: Add runtime support for distinct aggregation over grouped
> >>> windows (pending PR, under review)
> >>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> >>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
> >>> review)
> >>> - FLINK-6196: Support dynamic schema in Table Function (pending PR,
> under
> >>> review)
> >>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> >> (pending
> >>> PR, no review)
> >>> - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
> >>> Connector (pending PR, under review)
> >>> Important (closed):
> >>> - FLINK-6377: Support map types in the Table / SQL API (merged)
> >>> - FLINK-6398: RowSerializer's duplicate should always return a new
> >> instance
> >>> (merged)
> >>>
> >>>
> >>> I'll now collect the list of finished features in 1.3 and the blockers
> >> and
> >>> update above list.
> >>>
> >>> Stephan's email just came in, and I like the idea of freezing the
> feature
> >>> set now but extending the deadline to Friday for the branching. (Friday
> >> 3pm
> >>> CEST)
> >>>
> >>> Does everybody agree to that proposal?
> >>>
> >>>
> >>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org>
> wrote:
> >>>
> >>>> Thanks all for the lively discussion about the feature freeze and how
> to
> >>>> proceed.
> >>>> Because we committed to a time-based release schedule, we should not
> >> break
> >>>> the feature freeze too badly, or we would just disable the
> >>>> time-based-release-policy at the very first time it would trigger.
> >>>>
> >>>> Here are a few thoughts about what we can do:
> >>>>
> >>>> - First of all, please note that feature freeze does not mean bug fix
> >>>> freeze. Important bug fixes can and should go in as part of the 1.3.0
> >>>> release testing period
> >>>>
> >>>> - We should probably not add brand new features to the list at this
> >> point
> >>>> (that would most likely break the release schedule completely)
> >>>>
> >>>> - I think there is a case to include some features which are partially
> >>>> (but not fully) in master already, or exist basically completely as
> Pull
> >>>> Requests at this time.
> >>>>
> >>>> How about we do *freeze the set of features* for the release today,
> but
> >>>> *leave
> >>>> a grace period to get the complete-but-not-merged features in until
> the
> >> end
> >>>> of this week*?
> >>>> That would mean forking the release-1.3 branch end of this week.
> >>>>
> >>>>
> >>>> We should try and agree on which features fall under that list, and
> make
> >>>> sure each one has a committer committed to merging them in the next
> >> days.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
> >> tzulitai@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> I have 2 parts to discuss about delaying the feature freeze for a
> bit:
> >>>>>
> >>>>> 1. Allow upgrades to state serializers (FLINK-6178)
> >>>>> The feature is split up into 2 PRs, one of which is already open and
> >> the
> >>>>> second one coming up very soon.
> >>>>> PR #1 reconfigurable TypeSerializers: https://
> >>>>> github.com/apache/flink/pull/3804
> >>>>> PR #2 (will open soon) integrate serializer reconfiguration into
> state
> >>>>> restore flow (FLINK-6425)
> >>>>>
> >>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
> >>>>> Kafka - partition & topic regex discovery (PR -
> >>>> https://github.com/apache/
> >>>>> flink/pull/3746)
> >>>>> Kinesis - rescalable Kinesis consumer (PR -
> https://github.com/apache/
> >>>>> flink/pull/3001)
> >>>>>
> >>>>> All above mentioned features are not yet merged to master. I think an
> >>>>> additional few days to the end of this week for the feature freeze
> >> should
> >>>>> make this possible.
> >>>>>
> >>>>> Cheers,
> >>>>> Gordon
> >>>>>
> >>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org
> )
> >>>>> wrote:
> >>>>>
> >>>>> FLINK-5892 has been merged.
> >>>>>
> >>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also
> >> still
> >>>>> open and could use a second pair of eyes.
> >>>>>
> >>>>> On 28.04.2017 17:03, Kurt Young wrote:
> >>>>>> Hi Flavio,
> >>>>>>
> >>>>>> I have also fix the issue in 1.2 branch, but the next release will
> be
> >>>>> 1.2.2
> >>>>>>
> >>>>>> Best,
> >>>>>> Kurt
> >>>>>>
> >>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com>
> wrote:
> >>>>>>
> >>>>>>> Flavio:
> >>>>>>> Have you seen this (w.r.t. 1.2.1) ?
> >>>>>>>
> >>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> >>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> >>>>>>>
> >>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> >>>>> pompermaier@okkam.it>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink
> >>>> 1.2.0
> >>>>>>> in
> >>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer option
> in
> >>>> the
> >>>>>>>> short term..
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Flavio
> >>>>>>>>
> >>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> >>>> aljoscha@apache.org
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Ah, I see. The fix for that has been merged into master so it
> will
> >>>> be
> >>>>>>>>> release in Flink 1.3.
> >>>>>>>>>
> >>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> >>>> pompermaier@okkam.it>
> >>>>>>>>> wrote:
> >>>>>>>>>> Sorry, you're right Aljosha..the issue number is correct, the
> link
> >>>> is
> >>>>>>>>>> wrong! The correct one is https://issues.apache.org/
> >>>>>>>>> jira/browse/FLINK-6398
> >>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> >>>>>>>> aljoscha@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I think there might be a typo. We haven’t yet reached issue
> >> number
> >>>>>>>> 6389,
> >>>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> >>>>>>>>>>>
> >>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> >>>>>>> pompermaier@okkam.it>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>> If it's not a problem it will be great for us to include also
> >>>>>>>>> FLINK-6398
> >>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's
> not
> >> a
> >>>>>>> big
> >>>>>>>>>>> deal
> >>>>>>>>>>>> Best,
> >>>>>>>>>>>> Flavio
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> >>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Devs,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the release plan.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Could you also please add the feature FLINK-6196
> >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
> >>>>>>> dynamic
> >>>>>>>>>>> schema
> >>>>>>>>>>>>> in Table Function?
> >>>>>>>>>>>>> I’d like to update the code as comments left on PR today.
> >>>>>>>>>>>>> I will try to make sure the code is updated before the Apr
> >> 30th.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Zhuoluo 😀
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for starting this thread. It would be great to see the
> >>>>>>>>> following
> >>>>>>>>>>>>> features available in Flink 1.3:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> >>>>>>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
> >>>>>>>>> FLINK-6373
> >>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
> >> FLINK-6225
> >>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> All of them have gone through at least one round of review so
> >>>> I'm
> >>>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but
> it
> >>>>>>>> depends
> >>>>>>>>>>> on
> >>>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Haohui
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> >>>>>>>> chesnay@apache.org
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> >>>>>>> completion,
> >>>>>>>>> but
> >>>>>>>>>>>>> with only 1 day left before the weekend we're cutting it
> really
> >>>>>>>> short.
> >>>>>>>>>>>>> Since this eliminates a major pain point when updating jobs,
> as
> >>>> it
> >>>>>>>>>>>>> allows the modification of chains, another day or 2 would be
> >>>> good
> >>>>>>> i
> >>>>>>>>>>> think.
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>> Chesnay
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> >>>>>>> reporter)
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> release 1.3. It's in the final state of code review in
> >>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Bowen
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> >>>>>>>>>>>>> wangzhijiang999@aliyun.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>>>> Thank you for launching this topic!
> >>>>>>>>>>>>> I wish my latest refinement of buffer provider (
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
> >>>>>>> jobs
> >>>>>>>>> can
> >>>>>>>>>>>>> get benefit from it. And I think it can be completed with the
> >>>> help
> >>>>>>>> of
> >>>>>>>>>>>>> your
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> reviews this week.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,Zhijiang---------------------------------------------
> >>>>>>>>>>>>> ---------------------发件人：Ufuk
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >>>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rmetzger@apache.org
> >主
> >>>>>>>>>>>>> 题：[DISCUSS] Feature Freeze
> >>>>>>>>>>>>> Hey devs! :-)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We decided to follow a time-based release model with the
> >>>> upcoming
> >>>>>>>> 1.3
> >>>>>>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wanted to start a discussion to get a quick overview of the
> >>>>>>>> current
> >>>>>>>>>>>>> state of things.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
> >>>>>>>>>>>>> - Are there any major features we want in 1.3 that
> >>>>>>>>>>>>> have not been merged yet?
> >>>>>>>>>>>>> - Do we need to extend the feature freeze, because of an
> >>>>>>>>>>>>> important feature?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Would be great to gather a list of features/PRs that we want
> in
> >>>>>>> the
> >>>>>>>>>>>>> 1.3 release. This could be a good starting point for the
> >> release
> >>>>>>>>>>>>> manager (@Robert?).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ufuk
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Aljoscha Krettek <al...@apache.org>.

This is the list of blocking issues for Flink 1.3: https://issues.apache.org/jira/browse/FLINK-5740?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.3.0 <https://issues.apache.org/jira/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%20Blocker%20AND%20fixVersion%20=%201.3.0>

Could everyone please update the state of the issues mentioned in this thread to “blocking” and set the version to “1.3.0” if they feel that this is appropriate. Otherwise it is very hard for the release manager to get an overview over the progress of the release. With an up-to-date list it’s also possible to triage issues and discuss on the issue whether it really should be a release blocker.

Best,
Aljoscha
> On 2. May 2017, at 22:29, Eron Wright <er...@gmail.com> wrote:
> 
> Robert, I'd like to see FLINK-5974
> <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS support)
> added to the list of important issues.  A PR is ready.
> 
> On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <k....@data-artisans.com>
> wrote:
> 
>> The only thing that I want to add in the features to be added for 1.3
>> is the NOT pattern for the CEP library.
>> 
>> There is an open PR here: https://github.com/apache/flink/pull/3798 <
>> https://github.com/apache/flink/pull/3798>
>> which is not far from getting in.
>> 
>> Kostas
>> 
>>> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org> wrote:
>>> 
>>> Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
>>> freeze.
>>> 
>>> I didn't feature freeze yesterday (Monday) because it was a public
>> holiday
>>> here in Germany.
>>> 
>>> I haven't made up my mind whether to do the feature freeze today or not.
>>> Many important features seem to be close to completion.
>>> I don't think we can pick features and wait with the release until they
>> are
>>> finished. The only thing I can imagine we could do is extend the
>> deadline a
>>> little bit to give people more time to complete.
>>> But I'm strictly against multiple extensions.
>>> 
>>> To make the discussion a bit easier, I've collected all the JIRAs
>> mentioned
>>> in this thread. I decided to group them into "really important" and
>>> "important". I hope nobody is offended by my selection and I'm very open
>> to
>>> discuss it.
>>> It doesn't really matter anyways because we will not block the release on
>>> the completion of certain features.
>>> 
>>> 
>>> Completed Features for 1.3
>>> - TODO!
>>> 
>>> Blockers:
>>> - TODO!
>>> 
>>> 
>>> Really important (open):
>>> - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
>>> (pending PR)
>>> - FLINK-5906: Add support to register user defined aggregates in
>>> TableEnvironment (no PR)
>>> - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
>>> FLINK-6093)
>>> - FLINK-6093: Implement and turn on retraction for table sink (pending
>> PR)
>>> - FLINK-6334: Refactoring UDTF interface (pending PR, under review)
>>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under review)
>>> - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
>>> under review)
>>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no review
>> )
>>> 
>>> 
>>> Really important (closed):
>>> - FLINK-5892: Recover job state at the granularity of operator (merged)
>>> 
>>> 
>>> 
>>> Important (open):
>>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
>> review)
>>> - FLINK-6337: Remove the buffer provider from
>> PartitionRequestServerHandler
>>> (pending PR, under review)
>>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
>>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending
>> PR)
>>> - FLINK-6373: Add runtime support for distinct aggregation over grouped
>>> windows (pending PR, under review)
>>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
>>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
>>> review)
>>> - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
>>> review)
>>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
>> (pending
>>> PR, no review)
>>> - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
>>> Connector (pending PR, under review)
>>> Important (closed):
>>> - FLINK-6377: Support map types in the Table / SQL API (merged)
>>> - FLINK-6398: RowSerializer's duplicate should always return a new
>> instance
>>> (merged)
>>> 
>>> 
>>> I'll now collect the list of finished features in 1.3 and the blockers
>> and
>>> update above list.
>>> 
>>> Stephan's email just came in, and I like the idea of freezing the feature
>>> set now but extending the deadline to Friday for the branching. (Friday
>> 3pm
>>> CEST)
>>> 
>>> Does everybody agree to that proposal?
>>> 
>>> 
>>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org> wrote:
>>> 
>>>> Thanks all for the lively discussion about the feature freeze and how to
>>>> proceed.
>>>> Because we committed to a time-based release schedule, we should not
>> break
>>>> the feature freeze too badly, or we would just disable the
>>>> time-based-release-policy at the very first time it would trigger.
>>>> 
>>>> Here are a few thoughts about what we can do:
>>>> 
>>>> - First of all, please note that feature freeze does not mean bug fix
>>>> freeze. Important bug fixes can and should go in as part of the 1.3.0
>>>> release testing period
>>>> 
>>>> - We should probably not add brand new features to the list at this
>> point
>>>> (that would most likely break the release schedule completely)
>>>> 
>>>> - I think there is a case to include some features which are partially
>>>> (but not fully) in master already, or exist basically completely as Pull
>>>> Requests at this time.
>>>> 
>>>> How about we do *freeze the set of features* for the release today, but
>>>> *leave
>>>> a grace period to get the complete-but-not-merged features in until the
>> end
>>>> of this week*?
>>>> That would mean forking the release-1.3 branch end of this week.
>>>> 
>>>> 
>>>> We should try and agree on which features fall under that list, and make
>>>> sure each one has a committer committed to merging them in the next
>> days.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
>> tzulitai@apache.org>
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I have 2 parts to discuss about delaying the feature freeze for a bit:
>>>>> 
>>>>> 1. Allow upgrades to state serializers (FLINK-6178)
>>>>> The feature is split up into 2 PRs, one of which is already open and
>> the
>>>>> second one coming up very soon.
>>>>> PR #1 reconfigurable TypeSerializers: https://
>>>>> github.com/apache/flink/pull/3804
>>>>> PR #2 (will open soon) integrate serializer reconfiguration into state
>>>>> restore flow (FLINK-6425)
>>>>> 
>>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
>>>>> Kafka - partition & topic regex discovery (PR -
>>>> https://github.com/apache/
>>>>> flink/pull/3746)
>>>>> Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
>>>>> flink/pull/3001)
>>>>> 
>>>>> All above mentioned features are not yet merged to master. I think an
>>>>> additional few days to the end of this week for the feature freeze
>> should
>>>>> make this possible.
>>>>> 
>>>>> Cheers,
>>>>> Gordon
>>>>> 
>>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
>>>>> wrote:
>>>>> 
>>>>> FLINK-5892 has been merged.
>>>>> 
>>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also
>> still
>>>>> open and could use a second pair of eyes.
>>>>> 
>>>>> On 28.04.2017 17:03, Kurt Young wrote:
>>>>>> Hi Flavio,
>>>>>> 
>>>>>> I have also fix the issue in 1.2 branch, but the next release will be
>>>>> 1.2.2
>>>>>> 
>>>>>> Best,
>>>>>> Kurt
>>>>>> 
>>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
>>>>>> 
>>>>>>> Flavio:
>>>>>>> Have you seen this (w.r.t. 1.2.1) ?
>>>>>>> 
>>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
>>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
>>>>>>> 
>>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
>>>>> pompermaier@okkam.it>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink
>>>> 1.2.0
>>>>>>> in
>>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer option in
>>>> the
>>>>>>>> short term..
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Flavio
>>>>>>>> 
>>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
>>>> aljoscha@apache.org
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Ah, I see. The fix for that has been merged into master so it will
>>>> be
>>>>>>>>> release in Flink 1.3.
>>>>>>>>> 
>>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
>>>> pompermaier@okkam.it>
>>>>>>>>> wrote:
>>>>>>>>>> Sorry, you're right Aljosha..the issue number is correct, the link
>>>> is
>>>>>>>>>> wrong! The correct one is https://issues.apache.org/
>>>>>>>>> jira/browse/FLINK-6398
>>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
>>>>>>>> aljoscha@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I think there might be a typo. We haven’t yet reached issue
>> number
>>>>>>>> 6389,
>>>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
>>>>>>>>>>> 
>>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
>>>>>>> pompermaier@okkam.it>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> If it's not a problem it will be great for us to include also
>>>>>>>>> FLINK-6398
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not
>> a
>>>>>>> big
>>>>>>>>>>> deal
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Flavio
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
>>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Devs,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for the release plan.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Could you also please add the feature FLINK-6196
>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
>>>>>>> dynamic
>>>>>>>>>>> schema
>>>>>>>>>>>>> in Table Function?
>>>>>>>>>>>>> I’d like to update the code as comments left on PR today.
>>>>>>>>>>>>> I will try to make sure the code is updated before the Apr
>> 30th.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Zhuoluo 😀
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for starting this thread. It would be great to see the
>>>>>>>>> following
>>>>>>>>>>>>> features available in Flink 1.3:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
>>>>>>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
>>>>>>>>> FLINK-6373
>>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
>> FLINK-6225
>>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
>>>>>>>>>>>>> 
>>>>>>>>>>>>> All of them have gone through at least one round of review so
>>>> I'm
>>>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
>>>>>>>> depends
>>>>>>>>>>> on
>>>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Haohui
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
>>>>>>>> chesnay@apache.org
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
>>>>>>> completion,
>>>>>>>>> but
>>>>>>>>>>>>> with only 1 day left before the weekend we're cutting it really
>>>>>>>> short.
>>>>>>>>>>>>> Since this eliminates a major pain point when updating jobs, as
>>>> it
>>>>>>>>>>>>> allows the modification of chains, another day or 2 would be
>>>> good
>>>>>>> i
>>>>>>>>>>> think.
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Chesnay
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
>>>>>>> reporter)
>>>>>>>>>>>>> into
>>>>>>>>>>>>> 
>>>>>>>>>>>>> release 1.3. It's in the final state of code review in
>>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Bowen
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>>>>>>>>>>>>> wangzhijiang999@aliyun.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>>>> Thank you for launching this topic!
>>>>>>>>>>>>> I wish my latest refinement of buffer provider (
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>> 
>>>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
>>>>>>> jobs
>>>>>>>>> can
>>>>>>>>>>>>> get benefit from it. And I think it can be completed with the
>>>> help
>>>>>>>> of
>>>>>>>>>>>>> your
>>>>>>>>>>>>> 
>>>>>>>>>>>>> reviews this week.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Cheers,Zhijiang---------------------------------------------
>>>>>>>>>>>>> ---------------------发件人：Ufuk
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>>>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>>>>>>>>>>>>> 题：[DISCUSS] Feature Freeze
>>>>>>>>>>>>> Hey devs! :-)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> We decided to follow a time-based release model with the
>>>> upcoming
>>>>>>>> 1.3
>>>>>>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I wanted to start a discussion to get a quick overview of the
>>>>>>>> current
>>>>>>>>>>>>> state of things.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
>>>>>>>>>>>>> - Are there any major features we want in 1.3 that
>>>>>>>>>>>>> have not been merged yet?
>>>>>>>>>>>>> - Do we need to extend the feature freeze, because of an
>>>>>>>>>>>>> important feature?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Would be great to gather a list of features/PRs that we want in
>>>>>>> the
>>>>>>>>>>>>> 1.3 release. This could be a good starting point for the
>> release
>>>>>>>>>>>>> manager (@Robert?).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Ufuk
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Eron Wright <er...@gmail.com>.

Robert, I'd like to see FLINK-5974
<https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS support)
added to the list of important issues.  A PR is ready.

On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <k....@data-artisans.com>
wrote:

> The only thing that I want to add in the features to be added for 1.3
> is the NOT pattern for the CEP library.
>
> There is an open PR here: https://github.com/apache/flink/pull/3798 <
> https://github.com/apache/flink/pull/3798>
> which is not far from getting in.
>
> Kostas
>
> > On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org> wrote:
> >
> > Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
> > freeze.
> >
> > I didn't feature freeze yesterday (Monday) because it was a public
> holiday
> > here in Germany.
> >
> > I haven't made up my mind whether to do the feature freeze today or not.
> > Many important features seem to be close to completion.
> > I don't think we can pick features and wait with the release until they
> are
> > finished. The only thing I can imagine we could do is extend the
> deadline a
> > little bit to give people more time to complete.
> > But I'm strictly against multiple extensions.
> >
> > To make the discussion a bit easier, I've collected all the JIRAs
> mentioned
> > in this thread. I decided to group them into "really important" and
> > "important". I hope nobody is offended by my selection and I'm very open
> to
> > discuss it.
> > It doesn't really matter anyways because we will not block the release on
> > the completion of certain features.
> >
> >
> > Completed Features for 1.3
> > - TODO!
> >
> > Blockers:
> > - TODO!
> >
> >
> > Really important (open):
> > - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
> > (pending PR)
> > - FLINK-5906: Add support to register user defined aggregates in
> > TableEnvironment (no PR)
> > - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
> > FLINK-6093)
> > - FLINK-6093: Implement and turn on retraction for table sink (pending
> PR)
> > - FLINK-6334: Refactoring UDTF interface (pending PR, under review)
> > - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under review)
> > - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
> > under review)
> > - FLINK-6178: Allow upgrades to state serializers (pending PR, no review
> )
> >
> >
> > Really important (closed):
> > - FLINK-5892: Recover job state at the granularity of operator (merged)
> >
> >
> >
> > Important (open):
> > - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
> review)
> > - FLINK-6337: Remove the buffer provider from
> PartitionRequestServerHandler
> > (pending PR, under review)
> > - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> > - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending
> PR)
> > - FLINK-6373: Add runtime support for distinct aggregation over grouped
> > windows (pending PR, under review)
> > - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> > - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
> > review)
> > - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
> > review)
> > - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
> (pending
> > PR, no review)
> > - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
> > Connector (pending PR, under review)
> > Important (closed):
> > - FLINK-6377: Support map types in the Table / SQL API (merged)
> > - FLINK-6398: RowSerializer's duplicate should always return a new
> instance
> > (merged)
> >
> >
> > I'll now collect the list of finished features in 1.3 and the blockers
> and
> > update above list.
> >
> > Stephan's email just came in, and I like the idea of freezing the feature
> > set now but extending the deadline to Friday for the branching. (Friday
> 3pm
> > CEST)
> >
> > Does everybody agree to that proposal?
> >
> >
> > On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org> wrote:
> >
> >> Thanks all for the lively discussion about the feature freeze and how to
> >> proceed.
> >> Because we committed to a time-based release schedule, we should not
> break
> >> the feature freeze too badly, or we would just disable the
> >> time-based-release-policy at the very first time it would trigger.
> >>
> >> Here are a few thoughts about what we can do:
> >>
> >>  - First of all, please note that feature freeze does not mean bug fix
> >> freeze. Important bug fixes can and should go in as part of the 1.3.0
> >> release testing period
> >>
> >>  - We should probably not add brand new features to the list at this
> point
> >> (that would most likely break the release schedule completely)
> >>
> >>  - I think there is a case to include some features which are partially
> >> (but not fully) in master already, or exist basically completely as Pull
> >> Requests at this time.
> >>
> >> How about we do *freeze the set of features* for the release today, but
> >> *leave
> >> a grace period to get the complete-but-not-merged features in until the
> end
> >> of this week*?
> >> That would mean forking the release-1.3 branch end of this week.
> >>
> >>
> >> We should try and agree on which features fall under that list, and make
> >> sure each one has a committer committed to merging them in the next
> days.
> >>
> >>
> >>
> >>
> >>
> >> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
> tzulitai@apache.org>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> I have 2 parts to discuss about delaying the feature freeze for a bit:
> >>>
> >>> 1. Allow upgrades to state serializers (FLINK-6178)
> >>> The feature is split up into 2 PRs, one of which is already open and
> the
> >>> second one coming up very soon.
> >>> PR #1 reconfigurable TypeSerializers: https://
> >>> github.com/apache/flink/pull/3804
> >>> PR #2 (will open soon) integrate serializer reconfiguration into state
> >>> restore flow (FLINK-6425)
> >>>
> >>> 2. Major new functionality for the Kafka and Kinesis consumers.
> >>> Kafka - partition & topic regex discovery (PR -
> >> https://github.com/apache/
> >>> flink/pull/3746)
> >>> Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
> >>> flink/pull/3001)
> >>>
> >>> All above mentioned features are not yet merged to master. I think an
> >>> additional few days to the end of this week for the feature freeze
> should
> >>> make this possible.
> >>>
> >>> Cheers,
> >>> Gordon
> >>>
> >>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
> >>> wrote:
> >>>
> >>> FLINK-5892 has been merged.
> >>>
> >>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also
> still
> >>> open and could use a second pair of eyes.
> >>>
> >>> On 28.04.2017 17:03, Kurt Young wrote:
> >>>> Hi Flavio,
> >>>>
> >>>> I have also fix the issue in 1.2 branch, but the next release will be
> >>> 1.2.2
> >>>>
> >>>> Best,
> >>>> Kurt
> >>>>
> >>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
> >>>>
> >>>>> Flavio:
> >>>>> Have you seen this (w.r.t. 1.2.1) ?
> >>>>>
> >>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> >>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> >>>>>
> >>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> >>> pompermaier@okkam.it>
> >>>>> wrote:
> >>>>>
> >>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink
> >> 1.2.0
> >>>>> in
> >>>>>> production and maybe an upgrade to 1.2.1 would be a safer option in
> >> the
> >>>>>> short term..
> >>>>>>
> >>>>>> Best,
> >>>>>> Flavio
> >>>>>>
> >>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> >> aljoscha@apache.org
> >>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Ah, I see. The fix for that has been merged into master so it will
> >> be
> >>>>>>> release in Flink 1.3.
> >>>>>>>
> >>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> >> pompermaier@okkam.it>
> >>>>>>> wrote:
> >>>>>>>> Sorry, you're right Aljosha..the issue number is correct, the link
> >> is
> >>>>>>>> wrong! The correct one is https://issues.apache.org/
> >>>>>>> jira/browse/FLINK-6398
> >>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> >>>>>> aljoscha@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I think there might be a typo. We haven’t yet reached issue
> number
> >>>>>> 6389,
> >>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> >>>>>>>>>
> >>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> >>>>> pompermaier@okkam.it>
> >>>>>>>>> wrote:
> >>>>>>>>>> If it's not a problem it will be great for us to include also
> >>>>>>> FLINK-6398
> >>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not
> a
> >>>>> big
> >>>>>>>>> deal
> >>>>>>>>>> Best,
> >>>>>>>>>> Flavio
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> >>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Devs,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for the release plan.
> >>>>>>>>>>>
> >>>>>>>>>>> Could you also please add the feature FLINK-6196
> >>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
> >>>>> dynamic
> >>>>>>>>> schema
> >>>>>>>>>>> in Table Function?
> >>>>>>>>>>> I’d like to update the code as comments left on PR today.
> >>>>>>>>>>> I will try to make sure the code is updated before the Apr
> 30th.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>
> >>>>>>>>>>> Zhuoluo 😀
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>>>>>>>>>>
> >>>>>>>>>>> Hello,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for starting this thread. It would be great to see the
> >>>>>>> following
> >>>>>>>>>>> features available in Flink 1.3:
> >>>>>>>>>>>
> >>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> >>>>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
> >>>>>>> FLINK-6373
> >>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281,
> FLINK-6225
> >>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> >>>>>>>>>>>
> >>>>>>>>>>> All of them have gone through at least one round of review so
> >> I'm
> >>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> >>>>>>>>>>>
> >>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
> >>>>>> depends
> >>>>>>>>> on
> >>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Haohui
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> >>>>>> chesnay@apache.org
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hello,
> >>>>>>>>>>>
> >>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> >>>>> completion,
> >>>>>>> but
> >>>>>>>>>>> with only 1 day left before the weekend we're cutting it really
> >>>>>> short.
> >>>>>>>>>>> Since this eliminates a major pain point when updating jobs, as
> >> it
> >>>>>>>>>>> allows the modification of chains, another day or 2 would be
> >> good
> >>>>> i
> >>>>>>>>> think.
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Chesnay
> >>>>>>>>>>>
> >>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> >>>>> reporter)
> >>>>>>>>>>> into
> >>>>>>>>>>>
> >>>>>>>>>>> release 1.3. It's in the final state of code review in
> >>>>>>>>>>> https://github.com/apache/flink/pull/3736
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Bowen
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> >>>>>>>>>>> wangzhijiang999@aliyun.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi Ufuk,
> >>>>>>>>>>> Thank you for launching this topic!
> >>>>>>>>>>> I wish my latest refinement of buffer provider (
> >>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>
> >>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
> >>>>> jobs
> >>>>>>> can
> >>>>>>>>>>> get benefit from it. And I think it can be completed with the
> >> help
> >>>>>> of
> >>>>>>>>>>> your
> >>>>>>>>>>>
> >>>>>>>>>>> reviews this week.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,Zhijiang---------------------------------------------
> >>>>>>>>>>> ---------------------发件人：Ufuk
> >>>>>>>>>>>
> >>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> >>>>>>>>>>> 题：[DISCUSS] Feature Freeze
> >>>>>>>>>>> Hey devs! :-)
> >>>>>>>>>>>
> >>>>>>>>>>> We decided to follow a time-based release model with the
> >> upcoming
> >>>>>> 1.3
> >>>>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
> >>>>>>>>>>>
> >>>>>>>>>>> I wanted to start a discussion to get a quick overview of the
> >>>>>> current
> >>>>>>>>>>> state of things.
> >>>>>>>>>>>
> >>>>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
> >>>>>>>>>>> - Are there any major features we want in 1.3 that
> >>>>>>>>>>> have not been merged yet?
> >>>>>>>>>>> - Do we need to extend the feature freeze, because of an
> >>>>>>>>>>> important feature?
> >>>>>>>>>>>
> >>>>>>>>>>> Would be great to gather a list of features/PRs that we want in
> >>>>> the
> >>>>>>>>>>> 1.3 release. This could be a good starting point for the
> release
> >>>>>>>>>>> manager (@Robert?).
> >>>>>>>>>>>
> >>>>>>>>>>> Best,
> >>>>>>>>>>>
> >>>>>>>>>>> Ufuk
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>
> >>>
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Kostas Kloudas <k....@data-artisans.com>.

The only thing that I want to add in the features to be added for 1.3
is the NOT pattern for the CEP library. 

There is an open PR here: https://github.com/apache/flink/pull/3798 <https://github.com/apache/flink/pull/3798>
which is not far from getting in.

Kostas

> On May 2, 2017, at 12:10 PM, Robert Metzger <rm...@apache.org> wrote:
> 
> Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
> freeze.
> 
> I didn't feature freeze yesterday (Monday) because it was a public holiday
> here in Germany.
> 
> I haven't made up my mind whether to do the feature freeze today or not.
> Many important features seem to be close to completion.
> I don't think we can pick features and wait with the release until they are
> finished. The only thing I can imagine we could do is extend the deadline a
> little bit to give people more time to complete.
> But I'm strictly against multiple extensions.
> 
> To make the discussion a bit easier, I've collected all the JIRAs mentioned
> in this thread. I decided to group them into "really important" and
> "important". I hope nobody is offended by my selection and I'm very open to
> discuss it.
> It doesn't really matter anyways because we will not block the release on
> the completion of certain features.
> 
> 
> Completed Features for 1.3
> - TODO!
> 
> Blockers:
> - TODO!
> 
> 
> Really important (open):
> - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
> (pending PR)
> - FLINK-5906: Add support to register user defined aggregates in
> TableEnvironment (no PR)
> - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
> FLINK-6093)
> - FLINK-6093: Implement and turn on retraction for table sink (pending PR)
> - FLINK-6334: Refactoring UDTF interface (pending PR, under review)
> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under review)
> - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
> under review)
> - FLINK-6178: Allow upgrades to state serializers (pending PR, no review )
> 
> 
> Really important (closed):
> - FLINK-5892: Recover job state at the granularity of operator (merged)
> 
> 
> 
> Important (open):
> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under review)
> - FLINK-6337: Remove the buffer provider from PartitionRequestServerHandler
> (pending PR, under review)
> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending PR)
> - FLINK-6373: Add runtime support for distinct aggregation over grouped
> windows (pending PR, under review)
> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
> review)
> - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
> review)
> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer (pending
> PR, no review)
> - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
> Connector (pending PR, under review)
> Important (closed):
> - FLINK-6377: Support map types in the Table / SQL API (merged)
> - FLINK-6398: RowSerializer's duplicate should always return a new instance
> (merged)
> 
> 
> I'll now collect the list of finished features in 1.3 and the blockers and
> update above list.
> 
> Stephan's email just came in, and I like the idea of freezing the feature
> set now but extending the deadline to Friday for the branching. (Friday 3pm
> CEST)
> 
> Does everybody agree to that proposal?
> 
> 
> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org> wrote:
> 
>> Thanks all for the lively discussion about the feature freeze and how to
>> proceed.
>> Because we committed to a time-based release schedule, we should not break
>> the feature freeze too badly, or we would just disable the
>> time-based-release-policy at the very first time it would trigger.
>> 
>> Here are a few thoughts about what we can do:
>> 
>>  - First of all, please note that feature freeze does not mean bug fix
>> freeze. Important bug fixes can and should go in as part of the 1.3.0
>> release testing period
>> 
>>  - We should probably not add brand new features to the list at this point
>> (that would most likely break the release schedule completely)
>> 
>>  - I think there is a case to include some features which are partially
>> (but not fully) in master already, or exist basically completely as Pull
>> Requests at this time.
>> 
>> How about we do *freeze the set of features* for the release today, but
>> *leave
>> a grace period to get the complete-but-not-merged features in until the end
>> of this week*?
>> That would mean forking the release-1.3 branch end of this week.
>> 
>> 
>> We should try and agree on which features fall under that list, and make
>> sure each one has a committer committed to merging them in the next days.
>> 
>> 
>> 
>> 
>> 
>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>> 
>>> Hi,
>>> 
>>> I have 2 parts to discuss about delaying the feature freeze for a bit:
>>> 
>>> 1. Allow upgrades to state serializers (FLINK-6178)
>>> The feature is split up into 2 PRs, one of which is already open and the
>>> second one coming up very soon.
>>> PR #1 reconfigurable TypeSerializers: https://
>>> github.com/apache/flink/pull/3804
>>> PR #2 (will open soon) integrate serializer reconfiguration into state
>>> restore flow (FLINK-6425)
>>> 
>>> 2. Major new functionality for the Kafka and Kinesis consumers.
>>> Kafka - partition & topic regex discovery (PR -
>> https://github.com/apache/
>>> flink/pull/3746)
>>> Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
>>> flink/pull/3001)
>>> 
>>> All above mentioned features are not yet merged to master. I think an
>>> additional few days to the end of this week for the feature freeze should
>>> make this possible.
>>> 
>>> Cheers,
>>> Gordon
>>> 
>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
>>> wrote:
>>> 
>>> FLINK-5892 has been merged.
>>> 
>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also still
>>> open and could use a second pair of eyes.
>>> 
>>> On 28.04.2017 17:03, Kurt Young wrote:
>>>> Hi Flavio,
>>>> 
>>>> I have also fix the issue in 1.2 branch, but the next release will be
>>> 1.2.2
>>>> 
>>>> Best,
>>>> Kurt
>>>> 
>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
>>>> 
>>>>> Flavio:
>>>>> Have you seen this (w.r.t. 1.2.1) ?
>>>>> 
>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
>>>>> 
>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
>>> pompermaier@okkam.it>
>>>>> wrote:
>>>>> 
>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink
>> 1.2.0
>>>>> in
>>>>>> production and maybe an upgrade to 1.2.1 would be a safer option in
>> the
>>>>>> short term..
>>>>>> 
>>>>>> Best,
>>>>>> Flavio
>>>>>> 
>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
>> aljoscha@apache.org
>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> Ah, I see. The fix for that has been merged into master so it will
>> be
>>>>>>> release in Flink 1.3.
>>>>>>> 
>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
>> pompermaier@okkam.it>
>>>>>>> wrote:
>>>>>>>> Sorry, you're right Aljosha..the issue number is correct, the link
>> is
>>>>>>>> wrong! The correct one is https://issues.apache.org/
>>>>>>> jira/browse/FLINK-6398
>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
>>>>>> aljoscha@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> I think there might be a typo. We haven’t yet reached issue number
>>>>>> 6389,
>>>>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
>>>>>>>>> 
>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
>>>>> pompermaier@okkam.it>
>>>>>>>>> wrote:
>>>>>>>>>> If it's not a problem it will be great for us to include also
>>>>>>> FLINK-6398
>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a
>>>>> big
>>>>>>>>> deal
>>>>>>>>>> Best,
>>>>>>>>>> Flavio
>>>>>>>>>> 
>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi Devs,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for the release plan.
>>>>>>>>>>> 
>>>>>>>>>>> Could you also please add the feature FLINK-6196
>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
>>>>> dynamic
>>>>>>>>> schema
>>>>>>>>>>> in Table Function?
>>>>>>>>>>> I’d like to update the code as comments left on PR today.
>>>>>>>>>>> I will try to make sure the code is updated before the Apr 30th.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> 
>>>>>>>>>>> Zhuoluo 😀
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
>>>>>>>>>>> 
>>>>>>>>>>> Hello,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for starting this thread. It would be great to see the
>>>>>>> following
>>>>>>>>>>> features available in Flink 1.3:
>>>>>>>>>>> 
>>>>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
>>>>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
>>>>>>> FLINK-6373
>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
>>>>>>>>>>> 
>>>>>>>>>>> All of them have gone through at least one round of review so
>> I'm
>>>>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
>>>>>>>>>>> 
>>>>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
>>>>>> depends
>>>>>>>>> on
>>>>>>>>>>> FLINK-5884 so it might be a little bit tough.
>>>>>>>>>>> 
>>>>>>>>>>> Regards,
>>>>>>>>>>> Haohui
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
>>>>>> chesnay@apache.org
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hello,
>>>>>>>>>>> 
>>>>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
>>>>> completion,
>>>>>>> but
>>>>>>>>>>> with only 1 day left before the weekend we're cutting it really
>>>>>> short.
>>>>>>>>>>> Since this eliminates a major pain point when updating jobs, as
>> it
>>>>>>>>>>> allows the modification of chains, another day or 2 would be
>> good
>>>>> i
>>>>>>>>> think.
>>>>>>>>>>> Regards,
>>>>>>>>>>> Chesnay
>>>>>>>>>>> 
>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
>>>>> reporter)
>>>>>>>>>>> into
>>>>>>>>>>> 
>>>>>>>>>>> release 1.3. It's in the final state of code review in
>>>>>>>>>>> https://github.com/apache/flink/pull/3736
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Bowen
>>>>>>>>>>> 
>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>>>>>>>>>>> wangzhijiang999@aliyun.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>> Thank you for launching this topic!
>>>>>>>>>>> I wish my latest refinement of buffer provider (
>>>>>>>>>>> 
>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>> 
>>>>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
>>>>> jobs
>>>>>>> can
>>>>>>>>>>> get benefit from it. And I think it can be completed with the
>> help
>>>>>> of
>>>>>>>>>>> your
>>>>>>>>>>> 
>>>>>>>>>>> reviews this week.
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,Zhijiang---------------------------------------------
>>>>>>>>>>> ---------------------发件人：Ufuk
>>>>>>>>>>> 
>>>>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>>>>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>>>>>>>>>>> 题：[DISCUSS] Feature Freeze
>>>>>>>>>>> Hey devs! :-)
>>>>>>>>>>> 
>>>>>>>>>>> We decided to follow a time-based release model with the
>> upcoming
>>>>>> 1.3
>>>>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
>>>>>>>>>>> 
>>>>>>>>>>> I wanted to start a discussion to get a quick overview of the
>>>>>> current
>>>>>>>>>>> state of things.
>>>>>>>>>>> 
>>>>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
>>>>>>>>>>> - Are there any major features we want in 1.3 that
>>>>>>>>>>> have not been merged yet?
>>>>>>>>>>> - Do we need to extend the feature freeze, because of an
>>>>>>>>>>> important feature?
>>>>>>>>>>> 
>>>>>>>>>>> Would be great to gather a list of features/PRs that we want in
>>>>> the
>>>>>>>>>>> 1.3 release. This could be a good starting point for the release
>>>>>>>>>>> manager (@Robert?).
>>>>>>>>>>> 
>>>>>>>>>>> Best,
>>>>>>>>>>> 
>>>>>>>>>>> Ufuk
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>> 
>>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Robert Metzger <rm...@apache.org>.

Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
freeze.

I didn't feature freeze yesterday (Monday) because it was a public holiday
here in Germany.

I haven't made up my mind whether to do the feature freeze today or not.
Many important features seem to be close to completion.
I don't think we can pick features and wait with the release until they are
finished. The only thing I can imagine we could do is extend the deadline a
little bit to give people more time to complete.
But I'm strictly against multiple extensions.

To make the discussion a bit easier, I've collected all the JIRAs mentioned
in this thread. I decided to group them into "really important" and
"important". I hope nobody is offended by my selection and I'm very open to
discuss it.
It doesn't really matter anyways because we will not block the release on
the completion of certain features.


Completed Features for 1.3
- TODO!

Blockers:
- TODO!


Really important (open):
- FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
(pending PR)
- FLINK-5906: Add support to register user defined aggregates in
TableEnvironment (no PR)
- FLINK-6047: Add support for Retraction in Table API / SQL (depends on
FLINK-6093)
- FLINK-6093: Implement and turn on retraction for table sink (pending PR)
- FLINK-6334: Refactoring UDTF interface (pending PR, under review)
- FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under review)
- FLINK-4545: Flink automatically manages TM network buffer (pending PR,
under review)
- FLINK-6178: Allow upgrades to state serializers (pending PR, no review )


Really important (closed):
- FLINK-5892: Recover job state at the granularity of operator (merged)



Important (open):
- FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under review)
- FLINK-6337: Remove the buffer provider from PartitionRequestServerHandler
(pending PR, under review)
- FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
- FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending PR)
- FLINK-6373: Add runtime support for distinct aggregation over grouped
windows (pending PR, under review)
- FLINK-6281: Create TableSink for JDBC (pending PR, under review)
- FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
review)
- FLINK-6196: Support dynamic schema in Table Function (pending PR, under
review)
- FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer (pending
PR, no review)
- FLINK-4821: Implement rescalable non-partitioned state for Kinesis
Connector (pending PR, under review)
Important (closed):
- FLINK-6377: Support map types in the Table / SQL API (merged)
- FLINK-6398: RowSerializer's duplicate should always return a new instance
(merged)


I'll now collect the list of finished features in 1.3 and the blockers and
update above list.

Stephan's email just came in, and I like the idea of freezing the feature
set now but extending the deadline to Friday for the branching. (Friday 3pm
CEST)

Does everybody agree to that proposal?


On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <se...@apache.org> wrote:

> Thanks all for the lively discussion about the feature freeze and how to
> proceed.
> Because we committed to a time-based release schedule, we should not break
> the feature freeze too badly, or we would just disable the
> time-based-release-policy at the very first time it would trigger.
>
> Here are a few thoughts about what we can do:
>
>   - First of all, please note that feature freeze does not mean bug fix
> freeze. Important bug fixes can and should go in as part of the 1.3.0
> release testing period
>
>   - We should probably not add brand new features to the list at this point
> (that would most likely break the release schedule completely)
>
>   - I think there is a case to include some features which are partially
> (but not fully) in master already, or exist basically completely as Pull
> Requests at this time.
>
> How about we do *freeze the set of features* for the release today, but
> *leave
> a grace period to get the complete-but-not-merged features in until the end
> of this week*?
> That would mean forking the release-1.3 branch end of this week.
>
>
> We should try and agree on which features fall under that list, and make
> sure each one has a committer committed to merging them in the next days.
>
>
>
>
>
> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
> > Hi,
> >
> > I have 2 parts to discuss about delaying the feature freeze for a bit:
> >
> > 1. Allow upgrades to state serializers (FLINK-6178)
> > The feature is split up into 2 PRs, one of which is already open and the
> > second one coming up very soon.
> > PR #1 reconfigurable TypeSerializers: https://
> > github.com/apache/flink/pull/3804
> > PR #2 (will open soon) integrate serializer reconfiguration into state
> > restore flow (FLINK-6425)
> >
> > 2. Major new functionality for the Kafka and Kinesis consumers.
> > Kafka - partition & topic regex discovery (PR -
> https://github.com/apache/
> > flink/pull/3746)
> > Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
> > flink/pull/3001)
> >
> > All above mentioned features are not yet merged to master. I think an
> > additional few days to the end of this week for the feature freeze should
> > make this possible.
> >
> > Cheers,
> > Gordon
> >
> > On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
> > wrote:
> >
> > FLINK-5892 has been merged.
> >
> > For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also still
> > open and could use a second pair of eyes.
> >
> > On 28.04.2017 17:03, Kurt Young wrote:
> > > Hi Flavio,
> > >
> > > I have also fix the issue in 1.2 branch, but the next release will be
> > 1.2.2
> > >
> > > Best,
> > > Kurt
> > >
> > > On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
> > >
> > >> Flavio:
> > >> Have you seen this (w.r.t. 1.2.1) ?
> > >>
> > >> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> > >> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> > >>
> > >> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> > pompermaier@okkam.it>
> > >> wrote:
> > >>
> > >>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink
> 1.2.0
> > >> in
> > >>> production and maybe an upgrade to 1.2.1 would be a safer option in
> the
> > >>> short term..
> > >>>
> > >>> Best,
> > >>> Flavio
> > >>>
> > >>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
> aljoscha@apache.org
> > >
> > >>> wrote:
> > >>>
> > >>>> Ah, I see. The fix for that has been merged into master so it will
> be
> > >>>> release in Flink 1.3.
> > >>>>
> > >>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
> pompermaier@okkam.it>
> > >>>> wrote:
> > >>>>> Sorry, you're right Aljosha..the issue number is correct, the link
> is
> > >>>>> wrong! The correct one is https://issues.apache.org/
> > >>>> jira/browse/FLINK-6398
> > >>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> > >>> aljoscha@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> I think there might be a typo. We haven’t yet reached issue number
> > >>> 6389,
> > >>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> > >>>>>>
> > >>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> > >> pompermaier@okkam.it>
> > >>>>>> wrote:
> > >>>>>>> If it's not a problem it will be great for us to include also
> > >>>> FLINK-6398
> > >>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a
> > >> big
> > >>>>>> deal
> > >>>>>>> Best,
> > >>>>>>> Flavio
> > >>>>>>>
> > >>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> > >>>>>> zhuoluo.yzl@alibaba-inc.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Devs,
> > >>>>>>>>
> > >>>>>>>> Thanks for the release plan.
> > >>>>>>>>
> > >>>>>>>> Could you also please add the feature FLINK-6196
> > >>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
> > >> dynamic
> > >>>>>> schema
> > >>>>>>>> in Table Function?
> > >>>>>>>> I’d like to update the code as comments left on PR today.
> > >>>>>>>> I will try to make sure the code is updated before the Apr 30th.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>>
> > >>>>>>>> Zhuoluo 😀
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> > >>>>>>>>
> > >>>>>>>> Hello,
> > >>>>>>>>
> > >>>>>>>> Thanks for starting this thread. It would be great to see the
> > >>>> following
> > >>>>>>>> features available in Flink 1.3:
> > >>>>>>>>
> > >>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> > >>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
> > >>>> FLINK-6373
> > >>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> > >>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> > >>>>>>>>
> > >>>>>>>> All of them have gone through at least one round of review so
> I'm
> > >>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> > >>>>>>>>
> > >>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
> > >>> depends
> > >>>>>> on
> > >>>>>>>> FLINK-5884 so it might be a little bit tough.
> > >>>>>>>>
> > >>>>>>>> Regards,
> > >>>>>>>> Haohui
> > >>>>>>>>
> > >>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> > >>> chesnay@apache.org
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Hello,
> > >>>>>>>>
> > >>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> > >> completion,
> > >>>> but
> > >>>>>>>> with only 1 day left before the weekend we're cutting it really
> > >>> short.
> > >>>>>>>> Since this eliminates a major pain point when updating jobs, as
> it
> > >>>>>>>> allows the modification of chains, another day or 2 would be
> good
> > >> i
> > >>>>>> think.
> > >>>>>>>> Regards,
> > >>>>>>>> Chesnay
> > >>>>>>>>
> > >>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> > >>>>>>>>
> > >>>>>>>> Hi Ufuk,
> > >>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> > >> reporter)
> > >>>>>>>> into
> > >>>>>>>>
> > >>>>>>>> release 1.3. It's in the final state of code review in
> > >>>>>>>> https://github.com/apache/flink/pull/3736
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> Bowen
> > >>>>>>>>
> > >>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> > >>>>>>>> wangzhijiang999@aliyun.com> wrote:
> > >>>>>>>>
> > >>>>>>>> Hi Ufuk,
> > >>>>>>>> Thank you for launching this topic!
> > >>>>>>>> I wish my latest refinement of buffer provider (
> > >>>>>>>>
> > >>>>>>>> https://issues.apache.org/
> > >>>>>>>>
> > >>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
> > >> jobs
> > >>>> can
> > >>>>>>>> get benefit from it. And I think it can be completed with the
> help
> > >>> of
> > >>>>>>>> your
> > >>>>>>>>
> > >>>>>>>> reviews this week.
> > >>>>>>>>
> > >>>>>>>> Cheers,Zhijiang---------------------------------------------
> > >>>>>>>> ---------------------发件人：Ufuk
> > >>>>>>>>
> > >>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> > >>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> > >>>>>>>> 题：[DISCUSS] Feature Freeze
> > >>>>>>>> Hey devs! :-)
> > >>>>>>>>
> > >>>>>>>> We decided to follow a time-based release model with the
> upcoming
> > >>> 1.3
> > >>>>>>>> release and the planned feature freeze is on Monday, May 1st.
> > >>>>>>>>
> > >>>>>>>> I wanted to start a discussion to get a quick overview of the
> > >>> current
> > >>>>>>>> state of things.
> > >>>>>>>>
> > >>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
> > >>>>>>>> - Are there any major features we want in 1.3 that
> > >>>>>>>> have not been merged yet?
> > >>>>>>>> - Do we need to extend the feature freeze, because of an
> > >>>>>>>> important feature?
> > >>>>>>>>
> > >>>>>>>> Would be great to gather a list of features/PRs that we want in
> > >> the
> > >>>>>>>> 1.3 release. This could be a good starting point for the release
> > >>>>>>>> manager (@Robert?).
> > >>>>>>>>
> > >>>>>>>> Best,
> > >>>>>>>>
> > >>>>>>>> Ufuk
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> >
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Stephan Ewen <se...@apache.org>.

Thanks all for the lively discussion about the feature freeze and how to
proceed.
Because we committed to a time-based release schedule, we should not break
the feature freeze too badly, or we would just disable the
time-based-release-policy at the very first time it would trigger.

Here are a few thoughts about what we can do:

  - First of all, please note that feature freeze does not mean bug fix
freeze. Important bug fixes can and should go in as part of the 1.3.0
release testing period

  - We should probably not add brand new features to the list at this point
(that would most likely break the release schedule completely)

  - I think there is a case to include some features which are partially
(but not fully) in master already, or exist basically completely as Pull
Requests at this time.

How about we do *freeze the set of features* for the release today, but *leave
a grace period to get the complete-but-not-merged features in until the end
of this week*?
That would mean forking the release-1.3 branch end of this week.


We should try and agree on which features fall under that list, and make
sure each one has a committer committed to merging them in the next days.





On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi,
>
> I have 2 parts to discuss about delaying the feature freeze for a bit:
>
> 1. Allow upgrades to state serializers (FLINK-6178)
> The feature is split up into 2 PRs, one of which is already open and the
> second one coming up very soon.
> PR #1 reconfigurable TypeSerializers: https://
> github.com/apache/flink/pull/3804
> PR #2 (will open soon) integrate serializer reconfiguration into state
> restore flow (FLINK-6425)
>
> 2. Major new functionality for the Kafka and Kinesis consumers.
> Kafka - partition & topic regex discovery (PR - https://github.com/apache/
> flink/pull/3746)
> Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
> flink/pull/3001)
>
> All above mentioned features are not yet merged to master. I think an
> additional few days to the end of this week for the feature freeze should
> make this possible.
>
> Cheers,
> Gordon
>
> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
> wrote:
>
> FLINK-5892 has been merged.
>
> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also still
> open and could use a second pair of eyes.
>
> On 28.04.2017 17:03, Kurt Young wrote:
> > Hi Flavio,
> >
> > I have also fix the issue in 1.2 branch, but the next release will be
> 1.2.2
> >
> > Best,
> > Kurt
> >
> > On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> >> Flavio:
> >> Have you seen this (w.r.t. 1.2.1) ?
> >>
> >> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> >> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
> >>
> >> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
> pompermaier@okkam.it>
> >> wrote:
> >>
> >>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0
> >> in
> >>> production and maybe an upgrade to 1.2.1 would be a safer option in the
> >>> short term..
> >>>
> >>> Best,
> >>> Flavio
> >>>
> >>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <aljoscha@apache.org
> >
> >>> wrote:
> >>>
> >>>> Ah, I see. The fix for that has been merged into master so it will be
> >>>> release in Flink 1.3.
> >>>>
> >>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>
> >>>> wrote:
> >>>>> Sorry, you're right Aljosha..the issue number is correct, the link is
> >>>>> wrong! The correct one is https://issues.apache.org/
> >>>> jira/browse/FLINK-6398
> >>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> >>> aljoscha@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> I think there might be a typo. We haven’t yet reached issue number
> >>> 6389,
> >>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.
> >>>>>>
> >>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> >> pompermaier@okkam.it>
> >>>>>> wrote:
> >>>>>>> If it's not a problem it will be great for us to include also
> >>>> FLINK-6398
> >>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a
> >> big
> >>>>>> deal
> >>>>>>> Best,
> >>>>>>> Flavio
> >>>>>>>
> >>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> >>>>>> zhuoluo.yzl@alibaba-inc.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Devs,
> >>>>>>>>
> >>>>>>>> Thanks for the release plan.
> >>>>>>>>
> >>>>>>>> Could you also please add the feature FLINK-6196
> >>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
> >> dynamic
> >>>>>> schema
> >>>>>>>> in Table Function?
> >>>>>>>> I’d like to update the code as comments left on PR today.
> >>>>>>>> I will try to make sure the code is updated before the Apr 30th.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>> Zhuoluo 😀
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> Thanks for starting this thread. It would be great to see the
> >>>> following
> >>>>>>>> features available in Flink 1.3:
> >>>>>>>>
> >>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
> >>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
> >>>> FLINK-6373
> >>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> >>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
> >>>>>>>>
> >>>>>>>> All of them have gone through at least one round of review so I'm
> >>>>>>>> optimistic that they can make it to 1.3 in a day or two.
> >>>>>>>>
> >>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
> >>> depends
> >>>>>> on
> >>>>>>>> FLINK-5884 so it might be a little bit tough.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Haohui
> >>>>>>>>
> >>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> >>> chesnay@apache.org
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hello,
> >>>>>>>>
> >>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
> >> completion,
> >>>> but
> >>>>>>>> with only 1 day left before the weekend we're cutting it really
> >>> short.
> >>>>>>>> Since this eliminates a major pain point when updating jobs, as it
> >>>>>>>> allows the modification of chains, another day or 2 would be good
> >> i
> >>>>>> think.
> >>>>>>>> Regards,
> >>>>>>>> Chesnay
> >>>>>>>>
> >>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
> >>>>>>>>
> >>>>>>>> Hi Ufuk,
> >>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics
> >> reporter)
> >>>>>>>> into
> >>>>>>>>
> >>>>>>>> release 1.3. It's in the final state of code review in
> >>>>>>>> https://github.com/apache/flink/pull/3736
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Bowen
> >>>>>>>>
> >>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> >>>>>>>> wangzhijiang999@aliyun.com> wrote:
> >>>>>>>>
> >>>>>>>> Hi Ufuk,
> >>>>>>>> Thank you for launching this topic!
> >>>>>>>> I wish my latest refinement of buffer provider (
> >>>>>>>>
> >>>>>>>> https://issues.apache.org/
> >>>>>>>>
> >>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the
> >> jobs
> >>>> can
> >>>>>>>> get benefit from it. And I think it can be completed with the help
> >>> of
> >>>>>>>> your
> >>>>>>>>
> >>>>>>>> reviews this week.
> >>>>>>>>
> >>>>>>>> Cheers,Zhijiang---------------------------------------------
> >>>>>>>> ---------------------发件人：Ufuk
> >>>>>>>>
> >>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> >>>>>>>> 题：[DISCUSS] Feature Freeze
> >>>>>>>> Hey devs! :-)
> >>>>>>>>
> >>>>>>>> We decided to follow a time-based release model with the upcoming
> >>> 1.3
> >>>>>>>> release and the planned feature freeze is on Monday, May 1st.
> >>>>>>>>
> >>>>>>>> I wanted to start a discussion to get a quick overview of the
> >>> current
> >>>>>>>> state of things.
> >>>>>>>>
> >>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
> >>>>>>>> - Are there any major features we want in 1.3 that
> >>>>>>>> have not been merged yet?
> >>>>>>>> - Do we need to extend the feature freeze, because of an
> >>>>>>>> important feature?
> >>>>>>>>
> >>>>>>>> Would be great to gather a list of features/PRs that we want in
> >> the
> >>>>>>>> 1.3 release. This could be a good starting point for the release
> >>>>>>>> manager (@Robert?).
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>>
> >>>>>>>> Ufuk
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
>
>

Re: [DISCUSS] Feature Freeze

Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.

Hi,

I have 2 parts to discuss about delaying the feature freeze for a bit:

1. Allow upgrades to state serializers (FLINK-6178)
The feature is split up into 2 PRs, one of which is already open and the second one coming up very soon.
PR #1 reconfigurable TypeSerializers: https://github.com/apache/flink/pull/3804
PR #2 (will open soon) integrate serializer reconfiguration into state restore flow (FLINK-6425)

2. Major new functionality for the Kafka and Kinesis consumers.
Kafka - partition & topic regex discovery (PR - https://github.com/apache/flink/pull/3746)
Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/flink/pull/3001)

All above mentioned features are not yet merged to master. I think an additional few days to the end of this week for the feature freeze should make this possible.

Cheers,
Gordon

On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org) wrote:

FLINK-5892 has been merged.  

For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also still  
open and could use a second pair of eyes.  

On 28.04.2017 17:03, Kurt Young wrote:  
> Hi Flavio,  
>  
> I have also fix the issue in 1.2 branch, but the next release will be 1.2.2  
>  
> Best,  
> Kurt  
>  
> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:  
>  
>> Flavio:  
>> Have you seen this (w.r.t. 1.2.1) ?  
>>  
>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+  
>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+  
>>  
>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <po...@okkam.it>  
>> wrote:  
>>  
>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0  
>> in  
>>> production and maybe an upgrade to 1.2.1 would be a safer option in the  
>>> short term..  
>>>  
>>> Best,  
>>> Flavio  
>>>  
>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <al...@apache.org>  
>>> wrote:  
>>>  
>>>> Ah, I see. The fix for that has been merged into master so it will be  
>>>> release in Flink 1.3.  
>>>>  
>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>  
>>>> wrote:  
>>>>> Sorry, you're right Aljosha..the issue number is correct, the link is  
>>>>> wrong! The correct one is https://issues.apache.org/  
>>>> jira/browse/FLINK-6398  
>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <  
>>> aljoscha@apache.org>  
>>>>> wrote:  
>>>>>  
>>>>>> I think there might be a typo. We haven’t yet reached issue number  
>>> 6389,  
>>>>>> if I’m not mistaken. The latest as I’m writing this is 6410.  
>>>>>>  
>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <  
>> pompermaier@okkam.it>  
>>>>>> wrote:  
>>>>>>> If it's not a problem it will be great for us to include also  
>>>> FLINK-6398  
>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a  
>> big  
>>>>>> deal  
>>>>>>> Best,  
>>>>>>> Flavio  
>>>>>>>  
>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <  
>>>>>> zhuoluo.yzl@alibaba-inc.com>  
>>>>>>> wrote:  
>>>>>>>  
>>>>>>>> Hi Devs,  
>>>>>>>>  
>>>>>>>> Thanks for the release plan.  
>>>>>>>>  
>>>>>>>> Could you also please add the feature FLINK-6196  
>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support  
>> dynamic  
>>>>>> schema  
>>>>>>>> in Table Function?  
>>>>>>>> I’d like to update the code as comments left on PR today.  
>>>>>>>> I will try to make sure the code is updated before the Apr 30th.  
>>>>>>>>  
>>>>>>>>  
>>>>>>>> Thanks,  
>>>>>>>>  
>>>>>>>> Zhuoluo 😀  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：  
>>>>>>>>  
>>>>>>>> Hello,  
>>>>>>>>  
>>>>>>>> Thanks for starting this thread. It would be great to see the  
>>>> following  
>>>>>>>> features available in Flink 1.3:  
>>>>>>>>  
>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377  
>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,  
>>>> FLINK-6373  
>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225  
>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998  
>>>>>>>>  
>>>>>>>> All of them have gone through at least one round of review so I'm  
>>>>>>>> optimistic that they can make it to 1.3 in a day or two.  
>>>>>>>>  
>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it  
>>> depends  
>>>>>> on  
>>>>>>>> FLINK-5884 so it might be a little bit tough.  
>>>>>>>>  
>>>>>>>> Regards,  
>>>>>>>> Haohui  
>>>>>>>>  
>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <  
>>> chesnay@apache.org  
>>>>>>>> wrote:  
>>>>>>>>  
>>>>>>>> Hello,  
>>>>>>>>  
>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing  
>> completion,  
>>>> but  
>>>>>>>> with only 1 day left before the weekend we're cutting it really  
>>> short.  
>>>>>>>> Since this eliminates a major pain point when updating jobs, as it  
>>>>>>>> allows the modification of chains, another day or 2 would be good  
>> i  
>>>>>> think.  
>>>>>>>> Regards,  
>>>>>>>> Chesnay  
>>>>>>>>  
>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:  
>>>>>>>>  
>>>>>>>> Hi Ufuk,  
>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog Http metrics  
>> reporter)  
>>>>>>>> into  
>>>>>>>>  
>>>>>>>> release 1.3. It's in the final state of code review in  
>>>>>>>> https://github.com/apache/flink/pull/3736  
>>>>>>>>  
>>>>>>>> Thanks,  
>>>>>>>> Bowen  
>>>>>>>>  
>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <  
>>>>>>>> wangzhijiang999@aliyun.com> wrote:  
>>>>>>>>  
>>>>>>>> Hi Ufuk,  
>>>>>>>> Thank you for launching this topic!  
>>>>>>>> I wish my latest refinement of buffer provider (  
>>>>>>>>  
>>>>>>>> https://issues.apache.org/  
>>>>>>>>  
>>>>>>>> jira/browse/FLINK-6337) to be included in 1.3 and most of the  
>> jobs  
>>>> can  
>>>>>>>> get benefit from it. And I think it can be completed with the help  
>>> of  
>>>>>>>> your  
>>>>>>>>  
>>>>>>>> reviews this week.  
>>>>>>>>  
>>>>>>>> Cheers,Zhijiang---------------------------------------------  
>>>>>>>> ---------------------发件人：Ufuk  
>>>>>>>>  
>>>>>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <  
>>>>>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主  
>>>>>>>> 题：[DISCUSS] Feature Freeze  
>>>>>>>> Hey devs! :-)  
>>>>>>>>  
>>>>>>>> We decided to follow a time-based release model with the upcoming  
>>> 1.3  
>>>>>>>> release and the planned feature freeze is on Monday, May 1st.  
>>>>>>>>  
>>>>>>>> I wanted to start a discussion to get a quick overview of the  
>>> current  
>>>>>>>> state of things.  
>>>>>>>>  
>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)  
>>>>>>>> - Are there any major features we want in 1.3 that  
>>>>>>>> have not been merged yet?  
>>>>>>>> - Do we need to extend the feature freeze, because of an  
>>>>>>>> important feature?  
>>>>>>>>  
>>>>>>>> Would be great to gather a list of features/PRs that we want in  
>> the  
>>>>>>>> 1.3 release. This could be a good starting point for the release  
>>>>>>>> manager (@Robert?).  
>>>>>>>>  
>>>>>>>> Best,  
>>>>>>>>  
>>>>>>>> Ufuk  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>>>  
>>>>>>

Re: [DISCUSS] Feature Freeze

Posted by Chesnay Schepler <ch...@apache.org>.

FLINK-5892 has been merged.

For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also still 
open and could use a second pair of eyes.

On 28.04.2017 17:03, Kurt Young wrote:
> Hi Flavio,
>
> I have also fix the issue in 1.2 branch, but the next release will be 1.2.2
>
> Best,
> Kurt
>
> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Flavio:
>> Have you seen this (w.r.t. 1.2.1) ?
>>
>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
>>
>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <po...@okkam.it>
>> wrote:
>>
>>> Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0
>> in
>>> production and maybe an upgrade to 1.2.1 would be a safer option in the
>>> short term..
>>>
>>> Best,
>>> Flavio
>>>
>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <al...@apache.org>
>>> wrote:
>>>
>>>> Ah, I see. The fix for that has been merged into master so it will be
>>>> release in Flink 1.3.
>>>>
>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>
>>>> wrote:
>>>>> Sorry, you're right Aljosha..the issue number is correct, the link is
>>>>> wrong! The correct one is https://issues.apache.org/
>>>> jira/browse/FLINK-6398
>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
>>> aljoscha@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I think there might be a typo. We haven\u2019t yet reached issue number
>>> 6389,
>>>>>> if I\u2019m not mistaken. The latest as I\u2019m writing this is 6410.
>>>>>>
>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
>> pompermaier@okkam.it>
>>>>>> wrote:
>>>>>>> If it's not a problem it will be great for us to include also
>>>> FLINK-6398
>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a
>> big
>>>>>> deal
>>>>>>> Best,
>>>>>>> Flavio
>>>>>>>
>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
>>>>>> zhuoluo.yzl@alibaba-inc.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> Thanks for the release plan.
>>>>>>>>
>>>>>>>> Could you also please add the feature FLINK-6196
>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
>> dynamic
>>>>>> schema
>>>>>>>> in Table Function?
>>>>>>>> I\u2019d like to update the code as comments left on PR today.
>>>>>>>> I will try to make sure the code is updated before the Apr 30th.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Zhuoluo \U0001f600
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> \u5728 2017\u5e744\u670828\u65e5\uff0c\u4e0a\u53488:48\uff0cHaohui Mai <ri...@gmail.com> \u5199\u9053\uff1a
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> Thanks for starting this thread. It would be great to see the
>>>> following
>>>>>>>> features available in Flink 1.3:
>>>>>>>>
>>>>>>>> * Support for complex schema: FLINK-6033, FLINK-6377
>>>>>>>> * Various improvements on SQL over group windows: FLINK-6335,
>>>> FLINK-6373
>>>>>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
>>>>>>>>
>>>>>>>> All of them have gone through at least one round of review so I'm
>>>>>>>> optimistic that they can make it to 1.3 in a day or two.
>>>>>>>>
>>>>>>>> Additionally it would be great to see FLINK-6232 go in, but it
>>> depends
>>>>>> on
>>>>>>>> FLINK-5884 so it might be a little bit tough.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Haohui
>>>>>>>>
>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
>>> chesnay@apache.org
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> FLINK-5892 (Restoring state by operator) is also nearing
>> completion,
>>>> but
>>>>>>>> with only 1 day left before the weekend we're cutting it really
>>> short.
>>>>>>>> Since this eliminates a major pain point when updating jobs, as it
>>>>>>>> allows the modification of chains, another day or 2 would be good
>> i
>>>>>> think.
>>>>>>>> Regards,
>>>>>>>> Chesnay
>>>>>>>>
>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
>>>>>>>>
>>>>>>>> Hi Ufuk,
>>>>>>>>     I'd like to get FLINK-6013 (Adding Datadog Http metrics
>> reporter)
>>>>>>>> into
>>>>>>>>
>>>>>>>> release 1.3. It's in the final state of code review in
>>>>>>>> https://github.com/apache/flink/pull/3736
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Bowen
>>>>>>>>
>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>>>>>>>> wangzhijiang999@aliyun.com> wrote:
>>>>>>>>
>>>>>>>> Hi Ufuk,
>>>>>>>> Thank you for launching this topic!
>>>>>>>> I wish my latest refinement of buffer provider (
>>>>>>>>
>>>>>>>> https://issues.apache.org/
>>>>>>>>
>>>>>>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the
>> jobs
>>>> can
>>>>>>>> get benefit from it. And I think it can be completed with the help
>>> of
>>>>>>>> your
>>>>>>>>
>>>>>>>> reviews this week.
>>>>>>>>
>>>>>>>> Cheers,Zhijiang---------------------------------------------
>>>>>>>> ---------------------\u53d1\u4ef6\u4eba\uff1aUfuk
>>>>>>>>
>>>>>>>> Celebi <uc...@apache.org>\u53d1\u9001\u65f6\u95f4\uff1a2017\u5e744\u670827\u65e5(\u661f\u671f\u56db) 22:25\u6536\u4ef6\u4eba\uff1adev <
>>>>>>>> dev@flink.apache.org>\u6284 \u9001\uff1aRobert Metzger <rm...@apache.org>\u4e3b
>>>>>>>> \u9898\uff1a[DISCUSS] Feature Freeze
>>>>>>>> Hey devs! :-)
>>>>>>>>
>>>>>>>> We decided to follow a time-based release model with the upcoming
>>> 1.3
>>>>>>>> release and the planned feature freeze is on Monday, May 1st.
>>>>>>>>
>>>>>>>> I wanted to start a discussion to get a quick overview of the
>>> current
>>>>>>>> state of things.
>>>>>>>>
>>>>>>>> - Is everyone on track and aware of the feature freeze? ;)
>>>>>>>> - Are there any major features we want in 1.3 that
>>>>>>>> have not been merged yet?
>>>>>>>> - Do we need to extend the feature freeze, because of an
>>>>>>>> important feature?
>>>>>>>>
>>>>>>>> Would be great to gather a list of features/PRs that we want in
>> the
>>>>>>>> 1.3 release. This could be a good starting point for the release
>>>>>>>> manager (@Robert?).
>>>>>>>>
>>>>>>>> Best,
>>>>>>>>
>>>>>>>> Ufuk
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>

Re: [DISCUSS] Feature Freeze

Posted by Kurt Young <yk...@gmail.com>.

Hi Flavio,

I have also fix the issue in 1.2 branch, but the next release will be 1.2.2

Best,
Kurt

On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yu...@gmail.com> wrote:

> Flavio:
> Have you seen this (w.r.t. 1.2.1) ?
>
> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
>
> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <po...@okkam.it>
> wrote:
>
> > Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0
> in
> > production and maybe an upgrade to 1.2.1 would be a safer option in the
> > short term..
> >
> > Best,
> > Flavio
> >
> > On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> > > Ah, I see. The fix for that has been merged into master so it will be
> > > release in Flink 1.3.
> > >
> > > > On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>
> > > wrote:
> > > >
> > > > Sorry, you're right Aljosha..the issue number is correct, the link is
> > > > wrong! The correct one is https://issues.apache.org/
> > > jira/browse/FLINK-6398
> > > >
> > > > On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> > aljoscha@apache.org>
> > > > wrote:
> > > >
> > > >> I think there might be a typo. We haven’t yet reached issue number
> > 6389,
> > > >> if I’m not mistaken. The latest as I’m writing this is 6410.
> > > >>
> > > >>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <
> pompermaier@okkam.it>
> > > >> wrote:
> > > >>>
> > > >>> If it's not a problem it will be great for us to include also
> > > FLINK-6398
> > > >>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a
> big
> > > >> deal
> > > >>>
> > > >>> Best,
> > > >>> Flavio
> > > >>>
> > > >>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> > > >> zhuoluo.yzl@alibaba-inc.com>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Devs,
> > > >>>>
> > > >>>> Thanks for the release plan.
> > > >>>>
> > > >>>> Could you also please add the feature FLINK-6196
> > > >>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support
> dynamic
> > > >> schema
> > > >>>> in Table Function?
> > > >>>> I’d like to update the code as comments left on PR today.
> > > >>>> I will try to make sure the code is updated before the Apr 30th.
> > > >>>>
> > > >>>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Zhuoluo 😀
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> > > >>>>
> > > >>>> Hello,
> > > >>>>
> > > >>>> Thanks for starting this thread. It would be great to see the
> > > following
> > > >>>> features available in Flink 1.3:
> > > >>>>
> > > >>>> * Support for complex schema: FLINK-6033, FLINK-6377
> > > >>>> * Various improvements on SQL over group windows: FLINK-6335,
> > > FLINK-6373
> > > >>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> > > >>>> * Decoupling Flink and Hadoop: FLINK-5998
> > > >>>>
> > > >>>> All of them have gone through at least one round of review so I'm
> > > >>>> optimistic that they can make it to 1.3 in a day or two.
> > > >>>>
> > > >>>> Additionally it would be great to see FLINK-6232 go in, but it
> > depends
> > > >> on
> > > >>>> FLINK-5884 so it might be a little bit tough.
> > > >>>>
> > > >>>> Regards,
> > > >>>> Haohui
> > > >>>>
> > > >>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> > chesnay@apache.org
> > > >
> > > >>>> wrote:
> > > >>>>
> > > >>>> Hello,
> > > >>>>
> > > >>>> FLINK-5892 (Restoring state by operator) is also nearing
> completion,
> > > but
> > > >>>> with only 1 day left before the weekend we're cutting it really
> > short.
> > > >>>>
> > > >>>> Since this eliminates a major pain point when updating jobs, as it
> > > >>>> allows the modification of chains, another day or 2 would be good
> i
> > > >> think.
> > > >>>>
> > > >>>> Regards,
> > > >>>> Chesnay
> > > >>>>
> > > >>>> On 27.04.2017 18:55, Bowen Li wrote:
> > > >>>>
> > > >>>> Hi Ufuk,
> > > >>>>    I'd like to get FLINK-6013 (Adding Datadog Http metrics
> reporter)
> > > >>>>
> > > >>>> into
> > > >>>>
> > > >>>> release 1.3. It's in the final state of code review in
> > > >>>> https://github.com/apache/flink/pull/3736
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Bowen
> > > >>>>
> > > >>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> > > >>>> wangzhijiang999@aliyun.com> wrote:
> > > >>>>
> > > >>>> Hi Ufuk,
> > > >>>> Thank you for launching this topic!
> > > >>>> I wish my latest refinement of buffer provider (
> > > >>>>
> > > >>>> https://issues.apache.org/
> > > >>>>
> > > >>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the
> jobs
> > > can
> > > >>>> get benefit from it. And I think it can be completed with the help
> > of
> > > >>>>
> > > >>>> your
> > > >>>>
> > > >>>> reviews this week.
> > > >>>>
> > > >>>> Cheers,Zhijiang---------------------------------------------
> > > >>>> ---------------------发件人：Ufuk
> > > >>>>
> > > >>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> > > >>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> > > >>>> 题：[DISCUSS] Feature Freeze
> > > >>>> Hey devs! :-)
> > > >>>>
> > > >>>> We decided to follow a time-based release model with the upcoming
> > 1.3
> > > >>>> release and the planned feature freeze is on Monday, May 1st.
> > > >>>>
> > > >>>> I wanted to start a discussion to get a quick overview of the
> > current
> > > >>>> state of things.
> > > >>>>
> > > >>>> - Is everyone on track and aware of the feature freeze? ;)
> > > >>>> - Are there any major features we want in 1.3 that
> > > >>>> have not been merged yet?
> > > >>>> - Do we need to extend the feature freeze, because of an
> > > >>>> important feature?
> > > >>>>
> > > >>>> Would be great to gather a list of features/PRs that we want in
> the
> > > >>>> 1.3 release. This could be a good starting point for the release
> > > >>>> manager (@Robert?).
> > > >>>>
> > > >>>> Best,
> > > >>>>
> > > >>>> Ufuk
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>
> > > >>
> > >
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Ted Yu <yu...@gmail.com>.

Flavio:
Have you seen this (w.r.t. 1.2.1) ?

http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+

On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <po...@okkam.it>
wrote:

> Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0 in
> production and maybe an upgrade to 1.2.1 would be a safer option in the
> short term..
>
> Best,
> Flavio
>
> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Ah, I see. The fix for that has been merged into master so it will be
> > release in Flink 1.3.
> >
> > > On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>
> > wrote:
> > >
> > > Sorry, you're right Aljosha..the issue number is correct, the link is
> > > wrong! The correct one is https://issues.apache.org/
> > jira/browse/FLINK-6398
> > >
> > > On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <
> aljoscha@apache.org>
> > > wrote:
> > >
> > >> I think there might be a typo. We haven’t yet reached issue number
> 6389,
> > >> if I’m not mistaken. The latest as I’m writing this is 6410.
> > >>
> > >>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <po...@okkam.it>
> > >> wrote:
> > >>>
> > >>> If it's not a problem it will be great for us to include also
> > FLINK-6398
> > >>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big
> > >> deal
> > >>>
> > >>> Best,
> > >>> Flavio
> > >>>
> > >>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> > >> zhuoluo.yzl@alibaba-inc.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Devs,
> > >>>>
> > >>>> Thanks for the release plan.
> > >>>>
> > >>>> Could you also please add the feature FLINK-6196
> > >>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic
> > >> schema
> > >>>> in Table Function?
> > >>>> I’d like to update the code as comments left on PR today.
> > >>>> I will try to make sure the code is updated before the Apr 30th.
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>>
> > >>>> Zhuoluo 😀
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> Thanks for starting this thread. It would be great to see the
> > following
> > >>>> features available in Flink 1.3:
> > >>>>
> > >>>> * Support for complex schema: FLINK-6033, FLINK-6377
> > >>>> * Various improvements on SQL over group windows: FLINK-6335,
> > FLINK-6373
> > >>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> > >>>> * Decoupling Flink and Hadoop: FLINK-5998
> > >>>>
> > >>>> All of them have gone through at least one round of review so I'm
> > >>>> optimistic that they can make it to 1.3 in a day or two.
> > >>>>
> > >>>> Additionally it would be great to see FLINK-6232 go in, but it
> depends
> > >> on
> > >>>> FLINK-5884 so it might be a little bit tough.
> > >>>>
> > >>>> Regards,
> > >>>> Haohui
> > >>>>
> > >>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <
> chesnay@apache.org
> > >
> > >>>> wrote:
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> FLINK-5892 (Restoring state by operator) is also nearing completion,
> > but
> > >>>> with only 1 day left before the weekend we're cutting it really
> short.
> > >>>>
> > >>>> Since this eliminates a major pain point when updating jobs, as it
> > >>>> allows the modification of chains, another day or 2 would be good i
> > >> think.
> > >>>>
> > >>>> Regards,
> > >>>> Chesnay
> > >>>>
> > >>>> On 27.04.2017 18:55, Bowen Li wrote:
> > >>>>
> > >>>> Hi Ufuk,
> > >>>>    I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
> > >>>>
> > >>>> into
> > >>>>
> > >>>> release 1.3. It's in the final state of code review in
> > >>>> https://github.com/apache/flink/pull/3736
> > >>>>
> > >>>> Thanks,
> > >>>> Bowen
> > >>>>
> > >>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> > >>>> wangzhijiang999@aliyun.com> wrote:
> > >>>>
> > >>>> Hi Ufuk,
> > >>>> Thank you for launching this topic!
> > >>>> I wish my latest refinement of buffer provider (
> > >>>>
> > >>>> https://issues.apache.org/
> > >>>>
> > >>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs
> > can
> > >>>> get benefit from it. And I think it can be completed with the help
> of
> > >>>>
> > >>>> your
> > >>>>
> > >>>> reviews this week.
> > >>>>
> > >>>> Cheers,Zhijiang---------------------------------------------
> > >>>> ---------------------发件人：Ufuk
> > >>>>
> > >>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> > >>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> > >>>> 题：[DISCUSS] Feature Freeze
> > >>>> Hey devs! :-)
> > >>>>
> > >>>> We decided to follow a time-based release model with the upcoming
> 1.3
> > >>>> release and the planned feature freeze is on Monday, May 1st.
> > >>>>
> > >>>> I wanted to start a discussion to get a quick overview of the
> current
> > >>>> state of things.
> > >>>>
> > >>>> - Is everyone on track and aware of the feature freeze? ;)
> > >>>> - Are there any major features we want in 1.3 that
> > >>>> have not been merged yet?
> > >>>> - Do we need to extend the feature freeze, because of an
> > >>>> important feature?
> > >>>>
> > >>>> Would be great to gather a list of features/PRs that we want in the
> > >>>> 1.3 release. This could be a good starting point for the release
> > >>>> manager (@Robert?).
> > >>>>
> > >>>> Best,
> > >>>>
> > >>>> Ufuk
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>
> > >>
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Flavio Pompermaier <po...@okkam.it>.

Any chance to cherry-pick this also into 1.2.1? We're usign Flink 1.2.0 in
production and maybe an upgrade to 1.2.1 would be a safer option in the
short term..

Best,
Flavio

On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <al...@apache.org>
wrote:

> Ah, I see. The fix for that has been merged into master so it will be
> release in Flink 1.3.
>
> > On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it>
> wrote:
> >
> > Sorry, you're right Aljosha..the issue number is correct, the link is
> > wrong! The correct one is https://issues.apache.org/
> jira/browse/FLINK-6398
> >
> > On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> >> I think there might be a typo. We haven’t yet reached issue number 6389,
> >> if I’m not mistaken. The latest as I’m writing this is 6410.
> >>
> >>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <po...@okkam.it>
> >> wrote:
> >>>
> >>> If it's not a problem it will be great for us to include also
> FLINK-6398
> >>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big
> >> deal
> >>>
> >>> Best,
> >>> Flavio
> >>>
> >>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> >> zhuoluo.yzl@alibaba-inc.com>
> >>> wrote:
> >>>
> >>>> Hi Devs,
> >>>>
> >>>> Thanks for the release plan.
> >>>>
> >>>> Could you also please add the feature FLINK-6196
> >>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic
> >> schema
> >>>> in Table Function?
> >>>> I’d like to update the code as comments left on PR today.
> >>>> I will try to make sure the code is updated before the Apr 30th.
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Zhuoluo 😀
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>>>
> >>>> Hello,
> >>>>
> >>>> Thanks for starting this thread. It would be great to see the
> following
> >>>> features available in Flink 1.3:
> >>>>
> >>>> * Support for complex schema: FLINK-6033, FLINK-6377
> >>>> * Various improvements on SQL over group windows: FLINK-6335,
> FLINK-6373
> >>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> >>>> * Decoupling Flink and Hadoop: FLINK-5998
> >>>>
> >>>> All of them have gone through at least one round of review so I'm
> >>>> optimistic that they can make it to 1.3 in a day or two.
> >>>>
> >>>> Additionally it would be great to see FLINK-6232 go in, but it depends
> >> on
> >>>> FLINK-5884 so it might be a little bit tough.
> >>>>
> >>>> Regards,
> >>>> Haohui
> >>>>
> >>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <chesnay@apache.org
> >
> >>>> wrote:
> >>>>
> >>>> Hello,
> >>>>
> >>>> FLINK-5892 (Restoring state by operator) is also nearing completion,
> but
> >>>> with only 1 day left before the weekend we're cutting it really short.
> >>>>
> >>>> Since this eliminates a major pain point when updating jobs, as it
> >>>> allows the modification of chains, another day or 2 would be good i
> >> think.
> >>>>
> >>>> Regards,
> >>>> Chesnay
> >>>>
> >>>> On 27.04.2017 18:55, Bowen Li wrote:
> >>>>
> >>>> Hi Ufuk,
> >>>>    I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
> >>>>
> >>>> into
> >>>>
> >>>> release 1.3. It's in the final state of code review in
> >>>> https://github.com/apache/flink/pull/3736
> >>>>
> >>>> Thanks,
> >>>> Bowen
> >>>>
> >>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> >>>> wangzhijiang999@aliyun.com> wrote:
> >>>>
> >>>> Hi Ufuk,
> >>>> Thank you for launching this topic!
> >>>> I wish my latest refinement of buffer provider (
> >>>>
> >>>> https://issues.apache.org/
> >>>>
> >>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs
> can
> >>>> get benefit from it. And I think it can be completed with the help of
> >>>>
> >>>> your
> >>>>
> >>>> reviews this week.
> >>>>
> >>>> Cheers,Zhijiang---------------------------------------------
> >>>> ---------------------发件人：Ufuk
> >>>>
> >>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> >>>> 题：[DISCUSS] Feature Freeze
> >>>> Hey devs! :-)
> >>>>
> >>>> We decided to follow a time-based release model with the upcoming 1.3
> >>>> release and the planned feature freeze is on Monday, May 1st.
> >>>>
> >>>> I wanted to start a discussion to get a quick overview of the current
> >>>> state of things.
> >>>>
> >>>> - Is everyone on track and aware of the feature freeze? ;)
> >>>> - Are there any major features we want in 1.3 that
> >>>> have not been merged yet?
> >>>> - Do we need to extend the feature freeze, because of an
> >>>> important feature?
> >>>>
> >>>> Would be great to gather a list of features/PRs that we want in the
> >>>> 1.3 release. This could be a good starting point for the release
> >>>> manager (@Robert?).
> >>>>
> >>>> Best,
> >>>>
> >>>> Ufuk
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>
> >>
>

Re: [DISCUSS] Feature Freeze

Posted by Aljoscha Krettek <al...@apache.org>.

Ah, I see. The fix for that has been merged into master so it will be release in Flink 1.3.

> On 28. Apr 2017, at 13:50, Flavio Pompermaier <po...@okkam.it> wrote:
> 
> Sorry, you're right Aljosha..the issue number is correct, the link is
> wrong! The correct one is https://issues.apache.org/jira/browse/FLINK-6398
> 
> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <al...@apache.org>
> wrote:
> 
>> I think there might be a typo. We haven’t yet reached issue number 6389,
>> if I’m not mistaken. The latest as I’m writing this is 6410.
>> 
>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier <po...@okkam.it>
>> wrote:
>>> 
>>> If it's not a problem it will be great for us to include also FLINK-6398
>>> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big
>> deal
>>> 
>>> Best,
>>> Flavio
>>> 
>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
>> zhuoluo.yzl@alibaba-inc.com>
>>> wrote:
>>> 
>>>> Hi Devs,
>>>> 
>>>> Thanks for the release plan.
>>>> 
>>>> Could you also please add the feature FLINK-6196
>>>> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic
>> schema
>>>> in Table Function?
>>>> I’d like to update the code as comments left on PR today.
>>>> I will try to make sure the code is updated before the Apr 30th.
>>>> 
>>>> 
>>>> Thanks,
>>>> 
>>>> Zhuoluo 😀
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
>>>> 
>>>> Hello,
>>>> 
>>>> Thanks for starting this thread. It would be great to see the following
>>>> features available in Flink 1.3:
>>>> 
>>>> * Support for complex schema: FLINK-6033, FLINK-6377
>>>> * Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
>>>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
>>>> * Decoupling Flink and Hadoop: FLINK-5998
>>>> 
>>>> All of them have gone through at least one round of review so I'm
>>>> optimistic that they can make it to 1.3 in a day or two.
>>>> 
>>>> Additionally it would be great to see FLINK-6232 go in, but it depends
>> on
>>>> FLINK-5884 so it might be a little bit tough.
>>>> 
>>>> Regards,
>>>> Haohui
>>>> 
>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
>>>> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> FLINK-5892 (Restoring state by operator) is also nearing completion, but
>>>> with only 1 day left before the weekend we're cutting it really short.
>>>> 
>>>> Since this eliminates a major pain point when updating jobs, as it
>>>> allows the modification of chains, another day or 2 would be good i
>> think.
>>>> 
>>>> Regards,
>>>> Chesnay
>>>> 
>>>> On 27.04.2017 18:55, Bowen Li wrote:
>>>> 
>>>> Hi Ufuk,
>>>>    I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
>>>> 
>>>> into
>>>> 
>>>> release 1.3. It's in the final state of code review in
>>>> https://github.com/apache/flink/pull/3736
>>>> 
>>>> Thanks,
>>>> Bowen
>>>> 
>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>>>> wangzhijiang999@aliyun.com> wrote:
>>>> 
>>>> Hi Ufuk,
>>>> Thank you for launching this topic!
>>>> I wish my latest refinement of buffer provider (
>>>> 
>>>> https://issues.apache.org/
>>>> 
>>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
>>>> get benefit from it. And I think it can be completed with the help of
>>>> 
>>>> your
>>>> 
>>>> reviews this week.
>>>> 
>>>> Cheers,Zhijiang---------------------------------------------
>>>> ---------------------发件人：Ufuk
>>>> 
>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>>>> 题：[DISCUSS] Feature Freeze
>>>> Hey devs! :-)
>>>> 
>>>> We decided to follow a time-based release model with the upcoming 1.3
>>>> release and the planned feature freeze is on Monday, May 1st.
>>>> 
>>>> I wanted to start a discussion to get a quick overview of the current
>>>> state of things.
>>>> 
>>>> - Is everyone on track and aware of the feature freeze? ;)
>>>> - Are there any major features we want in 1.3 that
>>>> have not been merged yet?
>>>> - Do we need to extend the feature freeze, because of an
>>>> important feature?
>>>> 
>>>> Would be great to gather a list of features/PRs that we want in the
>>>> 1.3 release. This could be a good starting point for the release
>>>> manager (@Robert?).
>>>> 
>>>> Best,
>>>> 
>>>> Ufuk
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Flavio Pompermaier <po...@okkam.it>.

Sorry, you're right Aljosha..the issue number is correct, the link is
wrong! The correct one is https://issues.apache.org/jira/browse/FLINK-6398

On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek <al...@apache.org>
wrote:

> I think there might be a typo. We haven’t yet reached issue number 6389,
> if I’m not mistaken. The latest as I’m writing this is 6410.
>
> > On 28. Apr 2017, at 10:00, Flavio Pompermaier <po...@okkam.it>
> wrote:
> >
> > If it's not a problem it will be great for us to include also FLINK-6398
> > <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big
> deal
> >
> > Best,
> > Flavio
> >
> > On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <
> zhuoluo.yzl@alibaba-inc.com>
> > wrote:
> >
> >> Hi Devs,
> >>
> >> Thanks for the release plan.
> >>
> >> Could you also please add the feature FLINK-6196
> >> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic
> schema
> >> in Table Function?
> >> I’d like to update the code as comments left on PR today.
> >> I will try to make sure the code is updated before the Apr 30th.
> >>
> >>
> >> Thanks,
> >>
> >> Zhuoluo 😀
> >>
> >>
> >>
> >>
> >>
> >> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> >>
> >> Hello,
> >>
> >> Thanks for starting this thread. It would be great to see the following
> >> features available in Flink 1.3:
> >>
> >> * Support for complex schema: FLINK-6033, FLINK-6377
> >> * Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
> >> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> >> * Decoupling Flink and Hadoop: FLINK-5998
> >>
> >> All of them have gone through at least one round of review so I'm
> >> optimistic that they can make it to 1.3 in a day or two.
> >>
> >> Additionally it would be great to see FLINK-6232 go in, but it depends
> on
> >> FLINK-5884 so it might be a little bit tough.
> >>
> >> Regards,
> >> Haohui
> >>
> >> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
> >> wrote:
> >>
> >> Hello,
> >>
> >> FLINK-5892 (Restoring state by operator) is also nearing completion, but
> >> with only 1 day left before the weekend we're cutting it really short.
> >>
> >> Since this eliminates a major pain point when updating jobs, as it
> >> allows the modification of chains, another day or 2 would be good i
> think.
> >>
> >> Regards,
> >> Chesnay
> >>
> >> On 27.04.2017 18:55, Bowen Li wrote:
> >>
> >> Hi Ufuk,
> >>     I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
> >>
> >> into
> >>
> >> release 1.3. It's in the final state of code review in
> >> https://github.com/apache/flink/pull/3736
> >>
> >> Thanks,
> >> Bowen
> >>
> >> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> >> wangzhijiang999@aliyun.com> wrote:
> >>
> >> Hi Ufuk,
> >> Thank you for launching this topic!
> >> I wish my latest refinement of buffer provider (
> >>
> >> https://issues.apache.org/
> >>
> >> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
> >> get benefit from it. And I think it can be completed with the help of
> >>
> >> your
> >>
> >> reviews this week.
> >>
> >> Cheers,Zhijiang---------------------------------------------
> >> ---------------------发件人：Ufuk
> >>
> >> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> >> 题：[DISCUSS] Feature Freeze
> >> Hey devs! :-)
> >>
> >> We decided to follow a time-based release model with the upcoming 1.3
> >> release and the planned feature freeze is on Monday, May 1st.
> >>
> >> I wanted to start a discussion to get a quick overview of the current
> >> state of things.
> >>
> >> - Is everyone on track and aware of the feature freeze? ;)
> >> - Are there any major features we want in 1.3 that
> >> have not been merged yet?
> >> - Do we need to extend the feature freeze, because of an
> >> important feature?
> >>
> >> Would be great to gather a list of features/PRs that we want in the
> >> 1.3 release. This could be a good starting point for the release
> >> manager (@Robert?).
> >>
> >> Best,
> >>
> >> Ufuk
> >>
> >>
> >>
> >>
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Aljoscha Krettek <al...@apache.org>.

I think there might be a typo. We haven’t yet reached issue number 6389, if I’m not mistaken. The latest as I’m writing this is 6410.

> On 28. Apr 2017, at 10:00, Flavio Pompermaier <po...@okkam.it> wrote:
> 
> If it's not a problem it will be great for us to include also FLINK-6398
> <https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big deal
> 
> Best,
> Flavio
> 
> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <zh...@alibaba-inc.com>
> wrote:
> 
>> Hi Devs,
>> 
>> Thanks for the release plan.
>> 
>> Could you also please add the feature FLINK-6196
>> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic schema
>> in Table Function?
>> I’d like to update the code as comments left on PR today.
>> I will try to make sure the code is updated before the Apr 30th.
>> 
>> 
>> Thanks,
>> 
>> Zhuoluo 😀
>> 
>> 
>> 
>> 
>> 
>> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
>> 
>> Hello,
>> 
>> Thanks for starting this thread. It would be great to see the following
>> features available in Flink 1.3:
>> 
>> * Support for complex schema: FLINK-6033, FLINK-6377
>> * Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
>> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
>> * Decoupling Flink and Hadoop: FLINK-5998
>> 
>> All of them have gone through at least one round of review so I'm
>> optimistic that they can make it to 1.3 in a day or two.
>> 
>> Additionally it would be great to see FLINK-6232 go in, but it depends on
>> FLINK-5884 so it might be a little bit tough.
>> 
>> Regards,
>> Haohui
>> 
>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>> 
>> Hello,
>> 
>> FLINK-5892 (Restoring state by operator) is also nearing completion, but
>> with only 1 day left before the weekend we're cutting it really short.
>> 
>> Since this eliminates a major pain point when updating jobs, as it
>> allows the modification of chains, another day or 2 would be good i think.
>> 
>> Regards,
>> Chesnay
>> 
>> On 27.04.2017 18:55, Bowen Li wrote:
>> 
>> Hi Ufuk,
>>     I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
>> 
>> into
>> 
>> release 1.3. It's in the final state of code review in
>> https://github.com/apache/flink/pull/3736
>> 
>> Thanks,
>> Bowen
>> 
>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>> wangzhijiang999@aliyun.com> wrote:
>> 
>> Hi Ufuk,
>> Thank you for launching this topic!
>> I wish my latest refinement of buffer provider (
>> 
>> https://issues.apache.org/
>> 
>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
>> get benefit from it. And I think it can be completed with the help of
>> 
>> your
>> 
>> reviews this week.
>> 
>> Cheers,Zhijiang---------------------------------------------
>> ---------------------发件人：Ufuk
>> 
>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>> 题：[DISCUSS] Feature Freeze
>> Hey devs! :-)
>> 
>> We decided to follow a time-based release model with the upcoming 1.3
>> release and the planned feature freeze is on Monday, May 1st.
>> 
>> I wanted to start a discussion to get a quick overview of the current
>> state of things.
>> 
>> - Is everyone on track and aware of the feature freeze? ;)
>> - Are there any major features we want in 1.3 that
>> have not been merged yet?
>> - Do we need to extend the feature freeze, because of an
>> important feature?
>> 
>> Would be great to gather a list of features/PRs that we want in the
>> 1.3 release. This could be a good starting point for the release
>> manager (@Robert?).
>> 
>> Best,
>> 
>> Ufuk
>> 
>> 
>> 
>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Flavio Pompermaier <po...@okkam.it>.

If it's not a problem it will be great for us to include also FLINK-6398
<https://issues.apache.org/jira/browse/FLINK-6938> if it's not a big deal

Best,
Flavio

On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo Yang <zh...@alibaba-inc.com>
wrote:

> Hi Devs,
>
> Thanks for the release plan.
>
> Could you also please add the feature FLINK-6196
> <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic schema
> in Table Function?
> I’d like to update the code as comments left on PR today.
> I will try to make sure the code is updated before the Apr 30th.
>
>
> Thanks,
>
> Zhuoluo 😀
>
>
>
>
>
> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
>
> Hello,
>
> Thanks for starting this thread. It would be great to see the following
> features available in Flink 1.3:
>
> * Support for complex schema: FLINK-6033, FLINK-6377
> * Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> * Decoupling Flink and Hadoop: FLINK-5998
>
> All of them have gone through at least one round of review so I'm
> optimistic that they can make it to 1.3 in a day or two.
>
> Additionally it would be great to see FLINK-6232 go in, but it depends on
> FLINK-5884 so it might be a little bit tough.
>
> Regards,
> Haohui
>
> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> Hello,
>
> FLINK-5892 (Restoring state by operator) is also nearing completion, but
> with only 1 day left before the weekend we're cutting it really short.
>
> Since this eliminates a major pain point when updating jobs, as it
> allows the modification of chains, another day or 2 would be good i think.
>
> Regards,
> Chesnay
>
> On 27.04.2017 18:55, Bowen Li wrote:
>
> Hi Ufuk,
>      I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
>
> into
>
> release 1.3. It's in the final state of code review in
> https://github.com/apache/flink/pull/3736
>
> Thanks,
> Bowen
>
> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> wangzhijiang999@aliyun.com> wrote:
>
> Hi Ufuk,
> Thank you for launching this topic!
> I wish my latest refinement of buffer provider (
>
> https://issues.apache.org/
>
> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
> get benefit from it. And I think it can be completed with the help of
>
> your
>
> reviews this week.
>
> Cheers,Zhijiang---------------------------------------------
> ---------------------发件人：Ufuk
>
> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> 题：[DISCUSS] Feature Freeze
> Hey devs! :-)
>
> We decided to follow a time-based release model with the upcoming 1.3
> release and the planned feature freeze is on Monday, May 1st.
>
> I wanted to start a discussion to get a quick overview of the current
> state of things.
>
> - Is everyone on track and aware of the feature freeze? ;)
> - Are there any major features we want in 1.3 that
> have not been merged yet?
> - Do we need to extend the feature freeze, because of an
> important feature?
>
> Would be great to gather a list of features/PRs that we want in the
> 1.3 release. This could be a good starting point for the release
> manager (@Robert?).
>
> Best,
>
> Ufuk
>
>
>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Zhuoluo Yang <zh...@alibaba-inc.com>.

Hi Devs,

Thanks for the release plan.

Could you also please add the feature FLINK-6196 <https://issues.apache.org/jira/browse/FLINK-6196> Support dynamic schema in Table Function?
I’d like to update the code as comments left on PR today.
I will try to make sure the code is updated before the Apr 30th.


Thanks,

Zhuoluo 😀





> 在 2017年4月28日，上午8:48，Haohui Mai <ri...@gmail.com> 写道：
> 
> Hello,
> 
> Thanks for starting this thread. It would be great to see the following
> features available in Flink 1.3:
> 
> * Support for complex schema: FLINK-6033, FLINK-6377
> * Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
> * StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
> * Decoupling Flink and Hadoop: FLINK-5998
> 
> All of them have gone through at least one round of review so I'm
> optimistic that they can make it to 1.3 in a day or two.
> 
> Additionally it would be great to see FLINK-6232 go in, but it depends on
> FLINK-5884 so it might be a little bit tough.
> 
> Regards,
> Haohui
> 
> On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
> wrote:
> 
>> Hello,
>> 
>> FLINK-5892 (Restoring state by operator) is also nearing completion, but
>> with only 1 day left before the weekend we're cutting it really short.
>> 
>> Since this eliminates a major pain point when updating jobs, as it
>> allows the modification of chains, another day or 2 would be good i think.
>> 
>> Regards,
>> Chesnay
>> 
>> On 27.04.2017 18:55, Bowen Li wrote:
>>> Hi Ufuk,
>>>      I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
>> into
>>> release 1.3. It's in the final state of code review in
>>> https://github.com/apache/flink/pull/3736
>>> 
>>> Thanks,
>>> Bowen
>>> 
>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
>>> wangzhijiang999@aliyun.com> wrote:
>>> 
>>>> Hi Ufuk,
>>>> Thank you for launching this topic!
>>>> I wish my latest refinement of buffer provider (
>> https://issues.apache.org/
>>>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
>>>> get benefit from it. And I think it can be completed with the help of
>> your
>>>> reviews this week.
>>>> 
>> Cheers,Zhijiang------------------------------------------------------------------发件人：Ufuk
>>>> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
>>>> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
>>>> 题：[DISCUSS] Feature Freeze
>>>> Hey devs! :-)
>>>> 
>>>> We decided to follow a time-based release model with the upcoming 1.3
>>>> release and the planned feature freeze is on Monday, May 1st.
>>>> 
>>>> I wanted to start a discussion to get a quick overview of the current
>>>> state of things.
>>>> 
>>>> - Is everyone on track and aware of the feature freeze? ;)
>>>> - Are there any major features we want in 1.3 that
>>>> have not been merged yet?
>>>> - Do we need to extend the feature freeze, because of an
>>>> important feature?
>>>> 
>>>> Would be great to gather a list of features/PRs that we want in the
>>>> 1.3 release. This could be a good starting point for the release
>>>> manager (@Robert?).
>>>> 
>>>> Best,
>>>> 
>>>> Ufuk
>>>> 
>> 
>>

Re: [DISCUSS] Feature Freeze

Posted by Haohui Mai <ri...@gmail.com>.

Hello,

Thanks for starting this thread. It would be great to see the following
features available in Flink 1.3:

* Support for complex schema: FLINK-6033, FLINK-6377
* Various improvements on SQL over group windows: FLINK-6335, FLINK-6373
* StreamTableSink for JDBC and Cassandra: FLINK-6281, FLINK-6225
* Decoupling Flink and Hadoop: FLINK-5998

All of them have gone through at least one round of review so I'm
optimistic that they can make it to 1.3 in a day or two.

Additionally it would be great to see FLINK-6232 go in, but it depends on
FLINK-5884 so it might be a little bit tough.

Regards,
Haohui

On Thu, Apr 27, 2017 at 12:22 PM Chesnay Schepler <ch...@apache.org>
wrote:

> Hello,
>
> FLINK-5892 (Restoring state by operator) is also nearing completion, but
> with only 1 day left before the weekend we're cutting it really short.
>
> Since this eliminates a major pain point when updating jobs, as it
> allows the modification of chains, another day or 2 would be good i think.
>
> Regards,
> Chesnay
>
> On 27.04.2017 18:55, Bowen Li wrote:
> > Hi Ufuk,
> >       I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter)
> into
> > release 1.3. It's in the final state of code review in
> > https://github.com/apache/flink/pull/3736
> >
> > Thanks,
> > Bowen
> >
> > On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> > wangzhijiang999@aliyun.com> wrote:
> >
> >> Hi Ufuk,
> >> Thank you for launching this topic!
> >> I wish my latest refinement of buffer provider (
> https://issues.apache.org/
> >> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
> >> get benefit from it. And I think it can be completed with the help of
> your
> >> reviews this week.
> >>
> Cheers,Zhijiang------------------------------------------------------------------发件人：Ufuk
> >> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> >> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> >> 题：[DISCUSS] Feature Freeze
> >> Hey devs! :-)
> >>
> >> We decided to follow a time-based release model with the upcoming 1.3
> >> release and the planned feature freeze is on Monday, May 1st.
> >>
> >> I wanted to start a discussion to get a quick overview of the current
> >> state of things.
> >>
> >> - Is everyone on track and aware of the feature freeze? ;)
> >> - Are there any major features we want in 1.3 that
> >> have not been merged yet?
> >> - Do we need to extend the feature freeze, because of an
> >> important feature?
> >>
> >> Would be great to gather a list of features/PRs that we want in the
> >> 1.3 release. This could be a good starting point for the release
> >> manager (@Robert?).
> >>
> >> Best,
> >>
> >> Ufuk
> >>
>
>

Re: [DISCUSS] Feature Freeze

Posted by Chesnay Schepler <ch...@apache.org>.

Hello,

FLINK-5892 (Restoring state by operator) is also nearing completion, but 
with only 1 day left before the weekend we're cutting it really short.

Since this eliminates a major pain point when updating jobs, as it 
allows the modification of chains, another day or 2 would be good i think.

Regards,
Chesnay

On 27.04.2017 18:55, Bowen Li wrote:
> Hi Ufuk,
>       I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter) into
> release 1.3. It's in the final state of code review in
> https://github.com/apache/flink/pull/3736
>
> Thanks,
> Bowen
>
> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
> wangzhijiang999@aliyun.com> wrote:
>
>> Hi Ufuk,
>> Thank you for launching this topic!
>> I wish my latest refinement of buffer provider (https://issues.apache.org/
>> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
>> get benefit from it. And I think it can be completed with the help of your
>> reviews this week.
>> Cheers,Zhijiang------------------------------------------------------------------\u53d1\u4ef6\u4eba\uff1aUfuk
>> Celebi <uc...@apache.org>\u53d1\u9001\u65f6\u95f4\uff1a2017\u5e744\u670827\u65e5(\u661f\u671f\u56db) 22:25\u6536\u4ef6\u4eba\uff1adev <
>> dev@flink.apache.org>\u6284 \u9001\uff1aRobert Metzger <rm...@apache.org>\u4e3b
>> \u9898\uff1a[DISCUSS] Feature Freeze
>> Hey devs! :-)
>>
>> We decided to follow a time-based release model with the upcoming 1.3
>> release and the planned feature freeze is on Monday, May 1st.
>>
>> I wanted to start a discussion to get a quick overview of the current
>> state of things.
>>
>> - Is everyone on track and aware of the feature freeze? ;)
>> - Are there any major features we want in 1.3 that
>> have not been merged yet?
>> - Do we need to extend the feature freeze, because of an
>> important feature?
>>
>> Would be great to gather a list of features/PRs that we want in the
>> 1.3 release. This could be a good starting point for the release
>> manager (@Robert?).
>>
>> Best,
>>
>> Ufuk
>>

Re: [DISCUSS] Feature Freeze

Posted by Bowen Li <bo...@offerupnow.com>.

Hi Ufuk,
     I'd like to get FLINK-6013 (Adding Datadog Http metrics reporter) into
release 1.3. It's in the final state of code review in
https://github.com/apache/flink/pull/3736

Thanks,
Bowen

On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999) <
wangzhijiang999@aliyun.com> wrote:

> Hi Ufuk,
> Thank you for launching this topic!
> I wish my latest refinement of buffer provider (https://issues.apache.org/
> jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can
> get benefit from it. And I think it can be completed with the help of your
> reviews this week.
> Cheers,Zhijiang------------------------------------------------------------------发件人：Ufuk
> Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <
> dev@flink.apache.org>抄 送：Robert Metzger <rm...@apache.org>主
> 题：[DISCUSS] Feature Freeze
> Hey devs! :-)
>
> We decided to follow a time-based release model with the upcoming 1.3
> release and the planned feature freeze is on Monday, May 1st.
>
> I wanted to start a discussion to get a quick overview of the current
> state of things.
>
> - Is everyone on track and aware of the feature freeze? ;)
> - Are there any major features we want in 1.3 that
> have not been merged yet?
> - Do we need to extend the feature freeze, because of an
> important feature?
>
> Would be great to gather a list of features/PRs that we want in the
> 1.3 release. This could be a good starting point for the release
> manager (@Robert?).
>
> Best,
>
> Ufuk
>

回复：[DISCUSS] Feature Freeze

Posted by "Zhijiang(wangzhijiang999)" <wa...@aliyun.com>.

Hi Ufuk,
Thank you for launching this topic!
I wish my latest refinement of buffer provider (https://issues.apache.org/jira/browse/FLINK-6337)  to be included in 1.3 and most of the jobs can get benefit from it. And I think it can be completed with the help of your reviews this week.
Cheers,Zhijiang------------------------------------------------------------------发件人：Ufuk Celebi <uc...@apache.org>发送时间：2017年4月27日(星期四) 22:25收件人：dev <de...@flink.apache.org>抄　送：Robert Metzger <rm...@apache.org>主　题：[DISCUSS] Feature Freeze
Hey devs! :-)

We decided to follow a time-based release model with the upcoming 1.3
release and the planned feature freeze is on Monday, May 1st.

I wanted to start a discussion to get a quick overview of the current
state of things.

- Is everyone on track and aware of the feature freeze? ;)
- Are there any major features we want in 1.3 that have not been merged yet?
- Do we need to extend the feature freeze, because of an important feature?

Would be great to gather a list of features/PRs that we want in the
1.3 release. This could be a good starting point for the release
manager (@Robert?).

Best,

Ufuk

Re: [DISCUSS] Feature Freeze

Posted by Shaoxuan Wang <ws...@gmail.com>.

Hi Ufuk,
Thanks for the heads-up.

In terms of table API and SQL, I am hoping we can get the following
features (which have not completed merged yet) included in 1.3:

1. UDAGG (FLINK-5564),  I am working on the last PR (FLINK-5906) to close
this feature.
2. Retract (FLINK-6047), we are planning to merge retract feature-branch to
master-branch after FLINK-6093 is merged.
3. Refactor UDTF interface(FLINK-6334), we are planning to open the PR
today.

I feel we are very close to complete the above features (thanks for all the
reviews from Fabian and Timo). So It would be great if we can extend a few
days to freeze the 1.3.

Thanks,
Shaoxuan

On Thu, Apr 27, 2017 at 10:24 PM, Ufuk Celebi <uc...@apache.org> wrote:

> Hey devs! :-)
>
> We decided to follow a time-based release model with the upcoming 1.3
> release and the planned feature freeze is on Monday, May 1st.
>
> I wanted to start a discussion to get a quick overview of the current
> state of things.
>
> - Is everyone on track and aware of the feature freeze? ;)
> - Are there any major features we want in 1.3 that have not been merged
> yet?
> - Do we need to extend the feature freeze, because of an important feature?
>
> Would be great to gather a list of features/PRs that we want in the
> 1.3 release. This could be a good starting point for the release
> manager (@Robert?).
>
> Best,
>
> Ufuk
>

Re: [DISCUSS] Feature Freeze

Posted by Fabian Hueske <fh...@gmail.com>.

Hi everybody,

from an Table API / SQL point of view, three major features are not
completed yet:
- user-defined aggregation functions: Most of the preparation work is done.
There is one PR missing to actually register and analyze UDAGGs.
- retraction support: This feature was developed in a feature branch
(table-retraction) and is almost complete as well. There is one PR missing
for the conversion of Tables to DataStreams.
- time indicators: Timo is working on this feature and AFAIK, close to
finish it.

I know that the contributors for the UDAGG and retraction support are
working on their last PRs right now.

So, from the Table API / SQL point of view, we are very! close to finish
our efforts.
A few days more time before the feature freeze would help to ensure that
the Table API / SQL makes a huge step forward with Flink 1.3.

Cheers, Fabian


2017-04-27 16:44 GMT+02:00 Henry Saputra <he...@gmail.com>:

> The FLINK-6364 <https://issues.apache.org/jira/browse/FLINK-6364> seems
> need an accompanying FLIP [1] to help review.
>
> I dont see for this one in the list of existing proposals.
>
> - Henry
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/
> Flink+Improvement+Proposals
>
> On Thu, Apr 27, 2017 at 7:39 AM, Gyula Fóra <gy...@gmail.com> wrote:
>
> > Hi Ufuk,
> >
> > Thanks for starting this discussion!
> >
> > One feature that immediately comes to my mind is incremental
> checkpointing
> > given it's production impact.
> >
> > https://issues.apache.org/jira/browse/FLINK-6364
> >
> > It would be good to get some better understanding how the implementation
> > effort is going and whether it is reasonable to expect this to be
> included
> > in 1.3.
> >
> > Regards,
> > Gyula
> >
> > Ufuk Celebi <uc...@apache.org> ezt írta (időpont: 2017. ápr. 27., Cs,
> > 16:24):
> >
> > > Hey devs! :-)
> > >
> > > We decided to follow a time-based release model with the upcoming 1.3
> > > release and the planned feature freeze is on Monday, May 1st.
> > >
> > > I wanted to start a discussion to get a quick overview of the current
> > > state of things.
> > >
> > > - Is everyone on track and aware of the feature freeze? ;)
> > > - Are there any major features we want in 1.3 that have not been merged
> > > yet?
> > > - Do we need to extend the feature freeze, because of an important
> > feature?
> > >
> > > Would be great to gather a list of features/PRs that we want in the
> > > 1.3 release. This could be a good starting point for the release
> > > manager (@Robert?).
> > >
> > > Best,
> > >
> > > Ufuk
> > >
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Henry Saputra <he...@gmail.com>.

The FLINK-6364 <https://issues.apache.org/jira/browse/FLINK-6364> seems
need an accompanying FLIP [1] to help review.

I dont see for this one in the list of existing proposals.

- Henry

[1]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals

On Thu, Apr 27, 2017 at 7:39 AM, Gyula Fóra <gy...@gmail.com> wrote:

> Hi Ufuk,
>
> Thanks for starting this discussion!
>
> One feature that immediately comes to my mind is incremental checkpointing
> given it's production impact.
>
> https://issues.apache.org/jira/browse/FLINK-6364
>
> It would be good to get some better understanding how the implementation
> effort is going and whether it is reasonable to expect this to be included
> in 1.3.
>
> Regards,
> Gyula
>
> Ufuk Celebi <uc...@apache.org> ezt írta (időpont: 2017. ápr. 27., Cs,
> 16:24):
>
> > Hey devs! :-)
> >
> > We decided to follow a time-based release model with the upcoming 1.3
> > release and the planned feature freeze is on Monday, May 1st.
> >
> > I wanted to start a discussion to get a quick overview of the current
> > state of things.
> >
> > - Is everyone on track and aware of the feature freeze? ;)
> > - Are there any major features we want in 1.3 that have not been merged
> > yet?
> > - Do we need to extend the feature freeze, because of an important
> feature?
> >
> > Would be great to gather a list of features/PRs that we want in the
> > 1.3 release. This could be a good starting point for the release
> > manager (@Robert?).
> >
> > Best,
> >
> > Ufuk
> >
>

Re: [DISCUSS] Feature Freeze

Posted by Gyula Fóra <gy...@gmail.com>.

Hi Ufuk,

Thanks for starting this discussion!

One feature that immediately comes to my mind is incremental checkpointing
given it's production impact.

https://issues.apache.org/jira/browse/FLINK-6364

It would be good to get some better understanding how the implementation
effort is going and whether it is reasonable to expect this to be included
in 1.3.

Regards,
Gyula

Ufuk Celebi <uc...@apache.org> ezt írta (időpont: 2017. ápr. 27., Cs, 16:24):

> Hey devs! :-)
>
> We decided to follow a time-based release model with the upcoming 1.3
> release and the planned feature freeze is on Monday, May 1st.
>
> I wanted to start a discussion to get a quick overview of the current
> state of things.
>
> - Is everyone on track and aware of the feature freeze? ;)
> - Are there any major features we want in 1.3 that have not been merged
> yet?
> - Do we need to extend the feature freeze, because of an important feature?
>
> Would be great to gather a list of features/PRs that we want in the
> 1.3 release. This could be a good starting point for the release
> manager (@Robert?).
>
> Best,
>
> Ufuk
>