You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jingsong Li <ji...@gmail.com> on 2020/01/03 08:37:26 UTC
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Hi Jark,
+1 for default blink planner in SQL-CLI.
I believe this new planner can be put into practice in production.
We've worked hard for nearly a year, but the old planner didn't move on.
And I'd like to cc to user@flink.apache.org.
If anyone finds that blink planner has any significant defects and has a
larger regression than the old planner, please let us know. We will be very
grateful.
Best,
Jingsong Lee
On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
> +1 for this.
> We bring many SQL/API features and enhance stability in 1.10 release, and
> almost all of them happens in Blink planner.
> SQL CLI is the most convenient entrypoint for me, I believe many users
> will have a better experience If we set Blink planner as default planner.
>
> Best,
> Leonard
>
> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
> >
> > Since what blink planner can do is a superset of flink planner, big +1
> for changing the default planner to Blink planner from my side.
> >
> > Best,
> > Terry Wang
> >
> >
> >
> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
> >>
> >> Hi everyone,
> >>
> >> In 1.10 release, Flink SQL supports many awesome features and
> improvements,
> >> including:
> >> - support watermark statement and computed column in DDL
> >> - fully support all data types in Hive
> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
> >> - support INSERT OVERWRITE and INSERT PARTITION
> >>
> >> However, all the features and improvements are only avaiable in Blink
> >> planner, not in Old planner.
> >> There are also some other features are limited in Blink planner, e.g.
> >> Dimension Table Join [1],
> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
> so on.
> >>
> >> But Old planner is still the default planner in Table API & SQL. It is
> >> frustrating for users to set
> >> to blink planner manually when every time start a SQL CLI. And it's
> >> surprising to see unsupported
> >> exception if they trying out the new features but not switch planner.
> >>
> >> SQL CLI is a very important entrypoint for trying out new feautures and
> >> prototyping for users.
> >> In order to give new planner more exposures, I would like to suggest to
> set
> >> default planner
> >> for SQL Client to Blink planner before 1.10 release.
> >>
> >> The approach is just changing the default SQL CLI yaml
> configuration[5]. In
> >> this way, the existing
> >> environment is still compatible and unaffected.
> >>
> >> Changing the default planner for the whole Table API & SQL is another
> topic
> >> and is out of scope of this discussion.
> >>
> >> What do you think?
> >>
> >> Best,
> >> Jark
> >>
> >> [1]:
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
> >> [2]:
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
> >> [3]:
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
> >> [4]:
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
> >> [5]:
> >>
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
> >
>
>
--
Best, Jingsong Lee
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jark Wu <im...@gmail.com>.
Thanks everyone for the feedback!
We have reached a clear consensus that the proposal is accepted.
There are 3 binding votes from:
- Timo
- Hequn
- Jark
I will make sure that this change is applied to Flink code base and
created FLINK-15495 to track this.
Best,
Jark
On Mon, 6 Jan 2020 at 13:54, 贺小令 <go...@gmail.com> wrote:
> +1 for making the blink planner as default planner, the blink planner
> becomes more stable since 1.10
>
> Dian Fu <di...@gmail.com> 于2020年1月6日周一 上午11:51写道：
>
>> +1 to set blink planner as the default planner for SQL client considering
>> that so many features added since 1.10 are only available in the blink
>> planner.
>>
>> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>>
>> +1. I think it improves user experience.
>>
>> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
>>
>>> +1 for making blink planner as the default planner for SQL Client since
>>> we have made a huge improvement in 1.10.
>>>
>>> *Best Regards,*
>>> *Zhenghua Gao*
>>>
>>>
>>> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> We have used blink planner since 1.9.0 release in our production
>>>> environment, and it behaves really impressive.
>>>>
>>>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>>>
>>>>> +1 to make blink planner as the default planner for SQL Client, hence
>>>>> we can give the blink planner a bit more exposure.
>>>>>
>>>>> Best, Hequn
>>>>>
>>>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>>>
>>>>>> Hi Benoît,
>>>>>>
>>>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>>>> can target it into 1.9.2 and 1.10.
>>>>>>
>>>>>> Cheers,
>>>>>> Jark
>>>>>>
>>>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>>>
>>>>>>> > If anyone finds that blink planner has any significant defects
>>>>>>> and has a larger regression than the old planner, please let us know.
>>>>>>>
>>>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>>>> production are not covered:
>>>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>>>
>>>>>>> Cheers
>>>>>>> Ben
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1, I have already made blink as the default planner of flink
>>>>>>>> interpreter in Zeppelin
>>>>>>>>
>>>>>>>>
>>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>>>
>>>>>>>>> Hi Jark,
>>>>>>>>>
>>>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>>>> We've worked hard for nearly a year, but the old planner didn't
>>>>>>>>> move on.
>>>>>>>>>
>>>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>>>> be very grateful.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Jingsong Lee
>>>>>>>>>
>>>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for this.
>>>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>>>> planner.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Leonard
>>>>>>>>>>
>>>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>>>> >
>>>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>>>> >
>>>>>>>>>> > Best,
>>>>>>>>>> > Terry Wang
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>>>> >>
>>>>>>>>>> >> Hi everyone,
>>>>>>>>>> >>
>>>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>>>> improvements,
>>>>>>>>>> >> including:
>>>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>>>> >> - fully support all data types in Hive
>>>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>>>> >>
>>>>>>>>>> >> However, all the features and improvements are only avaiable
>>>>>>>>>> in Blink
>>>>>>>>>> >> planner, not in Old planner.
>>>>>>>>>> >> There are also some other features are limited in Blink
>>>>>>>>>> planner, e.g.
>>>>>>>>>> >> Dimension Table Join [1],
>>>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>>>> [4], and so on.
>>>>>>>>>> >>
>>>>>>>>>> >> But Old planner is still the default planner in Table API &
>>>>>>>>>> SQL. It is
>>>>>>>>>> >> frustrating for users to set
>>>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>>>> it's
>>>>>>>>>> >> surprising to see unsupported
>>>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>>>> planner.
>>>>>>>>>> >>
>>>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>>>> feautures and
>>>>>>>>>> >> prototyping for users.
>>>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>>>> suggest to set
>>>>>>>>>> >> default planner
>>>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>>>> >>
>>>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>>>> configuration[5]. In
>>>>>>>>>> >> this way, the existing
>>>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>>>> >>
>>>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>>>> another topic
>>>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>>>> >>
>>>>>>>>>> >> What do you think?
>>>>>>>>>> >>
>>>>>>>>>> >> Best,
>>>>>>>>>> >> Jark
>>>>>>>>>> >>
>>>>>>>>>> >> [1]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>>>> >> [2]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>>>> >> [3]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>>>> >> [4]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>>>> >> [5]:
>>>>>>>>>> >>
>>>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best, Jingsong Lee
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards
>>>>>>>>
>>>>>>>> Jeff Zhang
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Benoît Paris
>>>>>>> Ingénieur Machine Learning Explicable
>>>>>>> Tél : +33 6 60 74 23 00
>>>>>>> http://benoit.paris
>>>>>>> http://explicable.ml
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>>
>>>> Benchao Li
>>>> School of Electronics Engineering and Computer Science, Peking University
>>>> Tel:+86-15650713730
>>>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>>>
>>>>
>>
>> --
>> Best regards!
>> Rui Li
>>
>>
>>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jark Wu <im...@gmail.com>.
Thanks everyone for the feedback!
We have reached a clear consensus that the proposal is accepted.
There are 3 binding votes from:
- Timo
- Hequn
- Jark
I will make sure that this change is applied to Flink code base and
created FLINK-15495 to track this.
Best,
Jark
On Mon, 6 Jan 2020 at 13:54, 贺小令 <go...@gmail.com> wrote:
> +1 for making the blink planner as default planner, the blink planner
> becomes more stable since 1.10
>
> Dian Fu <di...@gmail.com> 于2020年1月6日周一 上午11:51写道：
>
>> +1 to set blink planner as the default planner for SQL client considering
>> that so many features added since 1.10 are only available in the blink
>> planner.
>>
>> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>>
>> +1. I think it improves user experience.
>>
>> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
>>
>>> +1 for making blink planner as the default planner for SQL Client since
>>> we have made a huge improvement in 1.10.
>>>
>>> *Best Regards,*
>>> *Zhenghua Gao*
>>>
>>>
>>> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> We have used blink planner since 1.9.0 release in our production
>>>> environment, and it behaves really impressive.
>>>>
>>>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>>>
>>>>> +1 to make blink planner as the default planner for SQL Client, hence
>>>>> we can give the blink planner a bit more exposure.
>>>>>
>>>>> Best, Hequn
>>>>>
>>>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>>>
>>>>>> Hi Benoît,
>>>>>>
>>>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>>>> can target it into 1.9.2 and 1.10.
>>>>>>
>>>>>> Cheers,
>>>>>> Jark
>>>>>>
>>>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>>>
>>>>>>> > If anyone finds that blink planner has any significant defects
>>>>>>> and has a larger regression than the old planner, please let us know.
>>>>>>>
>>>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>>>> production are not covered:
>>>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>>>
>>>>>>> Cheers
>>>>>>> Ben
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1, I have already made blink as the default planner of flink
>>>>>>>> interpreter in Zeppelin
>>>>>>>>
>>>>>>>>
>>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>>>
>>>>>>>>> Hi Jark,
>>>>>>>>>
>>>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>>>> We've worked hard for nearly a year, but the old planner didn't
>>>>>>>>> move on.
>>>>>>>>>
>>>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>>>> be very grateful.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Jingsong Lee
>>>>>>>>>
>>>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for this.
>>>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>>>> planner.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Leonard
>>>>>>>>>>
>>>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>>>> >
>>>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>>>> >
>>>>>>>>>> > Best,
>>>>>>>>>> > Terry Wang
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>>>> >>
>>>>>>>>>> >> Hi everyone,
>>>>>>>>>> >>
>>>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>>>> improvements,
>>>>>>>>>> >> including:
>>>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>>>> >> - fully support all data types in Hive
>>>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>>>> >>
>>>>>>>>>> >> However, all the features and improvements are only avaiable
>>>>>>>>>> in Blink
>>>>>>>>>> >> planner, not in Old planner.
>>>>>>>>>> >> There are also some other features are limited in Blink
>>>>>>>>>> planner, e.g.
>>>>>>>>>> >> Dimension Table Join [1],
>>>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>>>> [4], and so on.
>>>>>>>>>> >>
>>>>>>>>>> >> But Old planner is still the default planner in Table API &
>>>>>>>>>> SQL. It is
>>>>>>>>>> >> frustrating for users to set
>>>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>>>> it's
>>>>>>>>>> >> surprising to see unsupported
>>>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>>>> planner.
>>>>>>>>>> >>
>>>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>>>> feautures and
>>>>>>>>>> >> prototyping for users.
>>>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>>>> suggest to set
>>>>>>>>>> >> default planner
>>>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>>>> >>
>>>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>>>> configuration[5]. In
>>>>>>>>>> >> this way, the existing
>>>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>>>> >>
>>>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>>>> another topic
>>>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>>>> >>
>>>>>>>>>> >> What do you think?
>>>>>>>>>> >>
>>>>>>>>>> >> Best,
>>>>>>>>>> >> Jark
>>>>>>>>>> >>
>>>>>>>>>> >> [1]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>>>> >> [2]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>>>> >> [3]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>>>> >> [4]:
>>>>>>>>>> >>
>>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>>>> >> [5]:
>>>>>>>>>> >>
>>>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best, Jingsong Lee
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards
>>>>>>>>
>>>>>>>> Jeff Zhang
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Benoît Paris
>>>>>>> Ingénieur Machine Learning Explicable
>>>>>>> Tél : +33 6 60 74 23 00
>>>>>>> http://benoit.paris
>>>>>>> http://explicable.ml
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>>
>>>> Benchao Li
>>>> School of Electronics Engineering and Computer Science, Peking University
>>>> Tel:+86-15650713730
>>>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>>>
>>>>
>>
>> --
>> Best regards!
>> Rui Li
>>
>>
>>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by 贺小令 <go...@gmail.com>.
+1 for making the blink planner as default planner, the blink planner
becomes more stable since 1.10
Dian Fu <di...@gmail.com> 于2020年1月6日周一 上午11:51写道：
> +1 to set blink planner as the default planner for SQL client considering
> that so many features added since 1.10 are only available in the blink
> planner.
>
> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>
> +1. I think it improves user experience.
>
> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
>
>> +1 for making blink planner as the default planner for SQL Client since
>> we have made a huge improvement in 1.10.
>>
>> *Best Regards,*
>> *Zhenghua Gao*
>>
>>
>> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>>
>>> +1
>>>
>>> We have used blink planner since 1.9.0 release in our production
>>> environment, and it behaves really impressive.
>>>
>>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>>
>>>> +1 to make blink planner as the default planner for SQL Client, hence
>>>> we can give the blink planner a bit more exposure.
>>>>
>>>> Best, Hequn
>>>>
>>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Hi Benoît,
>>>>>
>>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>>> can target it into 1.9.2 and 1.10.
>>>>>
>>>>> Cheers,
>>>>> Jark
>>>>>
>>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>>
>>>>>> > If anyone finds that blink planner has any significant defects
>>>>>> and has a larger regression than the old planner, please let us know.
>>>>>>
>>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>>> production are not covered:
>>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>>
>>>>>> Cheers
>>>>>> Ben
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>>
>>>>>>> +1, I have already made blink as the default planner of flink
>>>>>>> interpreter in Zeppelin
>>>>>>>
>>>>>>>
>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>>
>>>>>>>> Hi Jark,
>>>>>>>>
>>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>>> We've worked hard for nearly a year, but the old planner didn't
>>>>>>>> move on.
>>>>>>>>
>>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>>> be very grateful.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Jingsong Lee
>>>>>>>>
>>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 for this.
>>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>>> planner.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Leonard
>>>>>>>>>
>>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>>> >
>>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>>> >
>>>>>>>>> > Best,
>>>>>>>>> > Terry Wang
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>>> >>
>>>>>>>>> >> Hi everyone,
>>>>>>>>> >>
>>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>>> improvements,
>>>>>>>>> >> including:
>>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>>> >> - fully support all data types in Hive
>>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>>> >>
>>>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>>>> Blink
>>>>>>>>> >> planner, not in Old planner.
>>>>>>>>> >> There are also some other features are limited in Blink
>>>>>>>>> planner, e.g.
>>>>>>>>> >> Dimension Table Join [1],
>>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>>> [4], and so on.
>>>>>>>>> >>
>>>>>>>>> >> But Old planner is still the default planner in Table API &
>>>>>>>>> SQL. It is
>>>>>>>>> >> frustrating for users to set
>>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>>> it's
>>>>>>>>> >> surprising to see unsupported
>>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>>> planner.
>>>>>>>>> >>
>>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>>> feautures and
>>>>>>>>> >> prototyping for users.
>>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>>> suggest to set
>>>>>>>>> >> default planner
>>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>>> >>
>>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>>> configuration[5]. In
>>>>>>>>> >> this way, the existing
>>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>>> >>
>>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>>> another topic
>>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>>> >>
>>>>>>>>> >> What do you think?
>>>>>>>>> >>
>>>>>>>>> >> Best,
>>>>>>>>> >> Jark
>>>>>>>>> >>
>>>>>>>>> >> [1]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>>> >> [2]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>>> >> [3]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>>> >> [4]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>>> >> [5]:
>>>>>>>>> >>
>>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best, Jingsong Lee
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards
>>>>>>>
>>>>>>> Jeff Zhang
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Benoît Paris
>>>>>> Ingénieur Machine Learning Explicable
>>>>>> Tél : +33 6 60 74 23 00
>>>>>> http://benoit.paris
>>>>>> http://explicable.ml
>>>>>>
>>>>>
>>>
>>> --
>>>
>>> Benchao Li
>>> School of Electronics Engineering and Computer Science, Peking University
>>> Tel:+86-15650713730
>>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>>
>>>
>
> --
> Best regards!
> Rui Li
>
>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by 贺小令 <go...@gmail.com>.
+1 for making the blink planner as default planner, the blink planner
becomes more stable since 1.10
Dian Fu <di...@gmail.com> 于2020年1月6日周一 上午11:51写道：
> +1 to set blink planner as the default planner for SQL client considering
> that so many features added since 1.10 are only available in the blink
> planner.
>
> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>
> +1. I think it improves user experience.
>
> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
>
>> +1 for making blink planner as the default planner for SQL Client since
>> we have made a huge improvement in 1.10.
>>
>> *Best Regards,*
>> *Zhenghua Gao*
>>
>>
>> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>>
>>> +1
>>>
>>> We have used blink planner since 1.9.0 release in our production
>>> environment, and it behaves really impressive.
>>>
>>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>>
>>>> +1 to make blink planner as the default planner for SQL Client, hence
>>>> we can give the blink planner a bit more exposure.
>>>>
>>>> Best, Hequn
>>>>
>>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Hi Benoît,
>>>>>
>>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>>> can target it into 1.9.2 and 1.10.
>>>>>
>>>>> Cheers,
>>>>> Jark
>>>>>
>>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>>
>>>>>> > If anyone finds that blink planner has any significant defects
>>>>>> and has a larger regression than the old planner, please let us know.
>>>>>>
>>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>>> production are not covered:
>>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>>
>>>>>> Cheers
>>>>>> Ben
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>>
>>>>>>> +1, I have already made blink as the default planner of flink
>>>>>>> interpreter in Zeppelin
>>>>>>>
>>>>>>>
>>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>>
>>>>>>>> Hi Jark,
>>>>>>>>
>>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>>> We've worked hard for nearly a year, but the old planner didn't
>>>>>>>> move on.
>>>>>>>>
>>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>>> be very grateful.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Jingsong Lee
>>>>>>>>
>>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 for this.
>>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>>> planner.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Leonard
>>>>>>>>>
>>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>>> >
>>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>>> >
>>>>>>>>> > Best,
>>>>>>>>> > Terry Wang
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>>> >>
>>>>>>>>> >> Hi everyone,
>>>>>>>>> >>
>>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>>> improvements,
>>>>>>>>> >> including:
>>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>>> >> - fully support all data types in Hive
>>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>>> >>
>>>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>>>> Blink
>>>>>>>>> >> planner, not in Old planner.
>>>>>>>>> >> There are also some other features are limited in Blink
>>>>>>>>> planner, e.g.
>>>>>>>>> >> Dimension Table Join [1],
>>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>>> [4], and so on.
>>>>>>>>> >>
>>>>>>>>> >> But Old planner is still the default planner in Table API &
>>>>>>>>> SQL. It is
>>>>>>>>> >> frustrating for users to set
>>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>>> it's
>>>>>>>>> >> surprising to see unsupported
>>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>>> planner.
>>>>>>>>> >>
>>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>>> feautures and
>>>>>>>>> >> prototyping for users.
>>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>>> suggest to set
>>>>>>>>> >> default planner
>>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>>> >>
>>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>>> configuration[5]. In
>>>>>>>>> >> this way, the existing
>>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>>> >>
>>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>>> another topic
>>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>>> >>
>>>>>>>>> >> What do you think?
>>>>>>>>> >>
>>>>>>>>> >> Best,
>>>>>>>>> >> Jark
>>>>>>>>> >>
>>>>>>>>> >> [1]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>>> >> [2]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>>> >> [3]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>>> >> [4]:
>>>>>>>>> >>
>>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>>> >> [5]:
>>>>>>>>> >>
>>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best, Jingsong Lee
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards
>>>>>>>
>>>>>>> Jeff Zhang
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Benoît Paris
>>>>>> Ingénieur Machine Learning Explicable
>>>>>> Tél : +33 6 60 74 23 00
>>>>>> http://benoit.paris
>>>>>> http://explicable.ml
>>>>>>
>>>>>
>>>
>>> --
>>>
>>> Benchao Li
>>> School of Electronics Engineering and Computer Science, Peking University
>>> Tel:+86-15650713730
>>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>>
>>>
>
> --
> Best regards!
> Rui Li
>
>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Dian Fu <di...@gmail.com>.
+1 to set blink planner as the default planner for SQL client considering that so many features added since 1.10 are only available in the blink planner.
> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>
> +1. I think it improves user experience.
>
> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <docete@gmail.com <ma...@gmail.com>> wrote:
> +1 for making blink planner as the default planner for SQL Client since we have made a huge improvement in 1.10.
>
> Best Regards,
> Zhenghua Gao
>
>
> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <libenchao@gmail.com <ma...@gmail.com>> wrote:
> +1
>
> We have used blink planner since 1.9.0 release in our production environment, and it behaves really impressive.
>
> Hequn Cheng <chenghequn@gmail.com <ma...@gmail.com>> 于2020年1月5日周日 下午1:58写道：
> +1 to make blink planner as the default planner for SQL Client, hence we can give the blink planner a bit more exposure.
>
> Best, Hequn
>
> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <imjark@gmail.com <ma...@gmail.com>> wrote:
> Hi Benoît,
>
> Thanks for the reminder. I will look into the issue and hopefully we can target it into 1.9.2 and 1.10.
>
> Cheers,
> Jark
>
> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <benoit.paris@centraliens-lille.org <ma...@centraliens-lille.org>> wrote:
> > If anyone finds that blink planner has any significant defects and has a larger regression than the old planner, please let us know.
>
> Overall, the Blink-exclusive features are must (TopN, deduplicate, LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in production are not covered:
> An edge case of Temporal Table Functions does not allow computed Tables (as opposed to TableSources) to be used on the query side in Blink (https://issues.apache.org/jira/browse/FLINK-14200 <https://issues.apache.org/jira/browse/FLINK-14200>)
>
> Cheers
> Ben
>
>
> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zjffdu@gmail.com <ma...@gmail.com>> wrote:
> +1, I have already made blink as the default planner of flink interpreter in Zeppelin
>
>
> Jingsong Li <jingsonglee0@gmail.com <ma...@gmail.com>> 于2020年1月3日周五 下午4:37写道：
> Hi Jark,
>
> +1 for default blink planner in SQL-CLI.
> I believe this new planner can be put into practice in production.
> We've worked hard for nearly a year, but the old planner didn't move on.
>
> And I'd like to cc to user@flink.apache.org <ma...@flink.apache.org>.
> If anyone finds that blink planner has any significant defects and has a larger regression than the old planner, please let us know. We will be very grateful.
>
> Best,
> Jingsong Lee
>
> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xbjtdcq@gmail.com <ma...@gmail.com>> wrote:
> +1 for this.
> We bring many SQL/API features and enhance stability in 1.10 release, and almost all of them happens in Blink planner.
> SQL CLI is the most convenient entrypoint for me, I believe many users will have a better experience If we set Blink planner as default planner.
>
> Best,
> Leonard
>
> > 在 2020年1月3日，15:16，Terry Wang <zjuwangg@gmail.com <ma...@gmail.com>> 写道：
> >
> > Since what blink planner can do is a superset of flink planner, big +1 for changing the default planner to Blink planner from my side.
> >
> > Best,
> > Terry Wang
> >
> >
> >
> >> 2020年1月3日 15:00，Jark Wu <imjark@gmail.com <ma...@gmail.com>> 写道：
> >>
> >> Hi everyone,
> >>
> >> In 1.10 release, Flink SQL supports many awesome features and improvements,
> >> including:
> >> - support watermark statement and computed column in DDL
> >> - fully support all data types in Hive
> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
> >> - support INSERT OVERWRITE and INSERT PARTITION
> >>
> >> However, all the features and improvements are only avaiable in Blink
> >> planner, not in Old planner.
> >> There are also some other features are limited in Blink planner, e.g.
> >> Dimension Table Join [1],
> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and so on.
> >>
> >> But Old planner is still the default planner in Table API & SQL. It is
> >> frustrating for users to set
> >> to blink planner manually when every time start a SQL CLI. And it's
> >> surprising to see unsupported
> >> exception if they trying out the new features but not switch planner.
> >>
> >> SQL CLI is a very important entrypoint for trying out new feautures and
> >> prototyping for users.
> >> In order to give new planner more exposures, I would like to suggest to set
> >> default planner
> >> for SQL Client to Blink planner before 1.10 release.
> >>
> >> The approach is just changing the default SQL CLI yaml configuration[5]. In
> >> this way, the existing
> >> environment is still compatible and unaffected.
> >>
> >> Changing the default planner for the whole Table API & SQL is another topic
> >> and is out of scope of this discussion.
> >>
> >> What do you think?
> >>
> >> Best,
> >> Jark
> >>
> >> [1]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table>
> >> [2]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n>
> >> [3]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication>
> >> [4]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html>
> >> [5]:
> >> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100 <https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100>
> >
>
>
>
> --
> Best, Jingsong Lee
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
> --
> Benoît Paris
> Ingénieur Machine Learning Explicable
> Tél : +33 6 60 74 23 00
> http://benoit.paris
> <http://benoit.paris/>
> http://explicable.ml
> <http://explicable.ml/>
>
> --
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com <ma...@gmail.com>; libenchao@pku.edu.cn <ma...@pku.edu.cn>
>
> --
> Best regards!
> Rui Li
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Dian Fu <di...@gmail.com>.
+1 to set blink planner as the default planner for SQL client considering that so many features added since 1.10 are only available in the blink planner.
> 在 2020年1月6日，上午11:04，Rui Li <li...@gmail.com> 写道：
>
> +1. I think it improves user experience.
>
> On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <docete@gmail.com <ma...@gmail.com>> wrote:
> +1 for making blink planner as the default planner for SQL Client since we have made a huge improvement in 1.10.
>
> Best Regards,
> Zhenghua Gao
>
>
> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <libenchao@gmail.com <ma...@gmail.com>> wrote:
> +1
>
> We have used blink planner since 1.9.0 release in our production environment, and it behaves really impressive.
>
> Hequn Cheng <chenghequn@gmail.com <ma...@gmail.com>> 于2020年1月5日周日 下午1:58写道：
> +1 to make blink planner as the default planner for SQL Client, hence we can give the blink planner a bit more exposure.
>
> Best, Hequn
>
> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <imjark@gmail.com <ma...@gmail.com>> wrote:
> Hi Benoît,
>
> Thanks for the reminder. I will look into the issue and hopefully we can target it into 1.9.2 and 1.10.
>
> Cheers,
> Jark
>
> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <benoit.paris@centraliens-lille.org <ma...@centraliens-lille.org>> wrote:
> > If anyone finds that blink planner has any significant defects and has a larger regression than the old planner, please let us know.
>
> Overall, the Blink-exclusive features are must (TopN, deduplicate, LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in production are not covered:
> An edge case of Temporal Table Functions does not allow computed Tables (as opposed to TableSources) to be used on the query side in Blink (https://issues.apache.org/jira/browse/FLINK-14200 <https://issues.apache.org/jira/browse/FLINK-14200>)
>
> Cheers
> Ben
>
>
> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zjffdu@gmail.com <ma...@gmail.com>> wrote:
> +1, I have already made blink as the default planner of flink interpreter in Zeppelin
>
>
> Jingsong Li <jingsonglee0@gmail.com <ma...@gmail.com>> 于2020年1月3日周五 下午4:37写道：
> Hi Jark,
>
> +1 for default blink planner in SQL-CLI.
> I believe this new planner can be put into practice in production.
> We've worked hard for nearly a year, but the old planner didn't move on.
>
> And I'd like to cc to user@flink.apache.org <ma...@flink.apache.org>.
> If anyone finds that blink planner has any significant defects and has a larger regression than the old planner, please let us know. We will be very grateful.
>
> Best,
> Jingsong Lee
>
> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xbjtdcq@gmail.com <ma...@gmail.com>> wrote:
> +1 for this.
> We bring many SQL/API features and enhance stability in 1.10 release, and almost all of them happens in Blink planner.
> SQL CLI is the most convenient entrypoint for me, I believe many users will have a better experience If we set Blink planner as default planner.
>
> Best,
> Leonard
>
> > 在 2020年1月3日，15:16，Terry Wang <zjuwangg@gmail.com <ma...@gmail.com>> 写道：
> >
> > Since what blink planner can do is a superset of flink planner, big +1 for changing the default planner to Blink planner from my side.
> >
> > Best,
> > Terry Wang
> >
> >
> >
> >> 2020年1月3日 15:00，Jark Wu <imjark@gmail.com <ma...@gmail.com>> 写道：
> >>
> >> Hi everyone,
> >>
> >> In 1.10 release, Flink SQL supports many awesome features and improvements,
> >> including:
> >> - support watermark statement and computed column in DDL
> >> - fully support all data types in Hive
> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
> >> - support INSERT OVERWRITE and INSERT PARTITION
> >>
> >> However, all the features and improvements are only avaiable in Blink
> >> planner, not in Old planner.
> >> There are also some other features are limited in Blink planner, e.g.
> >> Dimension Table Join [1],
> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and so on.
> >>
> >> But Old planner is still the default planner in Table API & SQL. It is
> >> frustrating for users to set
> >> to blink planner manually when every time start a SQL CLI. And it's
> >> surprising to see unsupported
> >> exception if they trying out the new features but not switch planner.
> >>
> >> SQL CLI is a very important entrypoint for trying out new feautures and
> >> prototyping for users.
> >> In order to give new planner more exposures, I would like to suggest to set
> >> default planner
> >> for SQL Client to Blink planner before 1.10 release.
> >>
> >> The approach is just changing the default SQL CLI yaml configuration[5]. In
> >> this way, the existing
> >> environment is still compatible and unaffected.
> >>
> >> Changing the default planner for the whole Table API & SQL is another topic
> >> and is out of scope of this discussion.
> >>
> >> What do you think?
> >>
> >> Best,
> >> Jark
> >>
> >> [1]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table>
> >> [2]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n>
> >> [3]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication>
> >> [4]:
> >> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html>
> >> [5]:
> >> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100 <https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100>
> >
>
>
>
> --
> Best, Jingsong Lee
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
> --
> Benoît Paris
> Ingénieur Machine Learning Explicable
> Tél : +33 6 60 74 23 00
> http://benoit.paris
> <http://benoit.paris/>
> http://explicable.ml
> <http://explicable.ml/>
>
> --
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com <ma...@gmail.com>; libenchao@pku.edu.cn <ma...@pku.edu.cn>
>
> --
> Best regards!
> Rui Li
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Rui Li <li...@gmail.com>.
+1. I think it improves user experience.
On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
> +1 for making blink planner as the default planner for SQL Client since we
> have made a huge improvement in 1.10.
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>
>> +1
>>
>> We have used blink planner since 1.9.0 release in our production
>> environment, and it behaves really impressive.
>>
>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>
>>> +1 to make blink planner as the default planner for SQL Client, hence we
>>> can give the blink planner a bit more exposure.
>>>
>>> Best, Hequn
>>>
>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>
>>>> Hi Benoît,
>>>>
>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>> can target it into 1.9.2 and 1.10.
>>>>
>>>> Cheers,
>>>> Jark
>>>>
>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>
>>>>> > If anyone finds that blink planner has any significant defects and
>>>>> has a larger regression than the old planner, please let us know.
>>>>>
>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>> production are not covered:
>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>
>>>>> Cheers
>>>>> Ben
>>>>>
>>>>>
>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>
>>>>>> +1, I have already made blink as the default planner of flink
>>>>>> interpreter in Zeppelin
>>>>>>
>>>>>>
>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>
>>>>>>> Hi Jark,
>>>>>>>
>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>>>> on.
>>>>>>>
>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>> be very grateful.
>>>>>>>
>>>>>>> Best,
>>>>>>> Jingsong Lee
>>>>>>>
>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1 for this.
>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>> planner.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Leonard
>>>>>>>>
>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>> >
>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>> >
>>>>>>>> > Best,
>>>>>>>> > Terry Wang
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>> >>
>>>>>>>> >> Hi everyone,
>>>>>>>> >>
>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>> improvements,
>>>>>>>> >> including:
>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>> >> - fully support all data types in Hive
>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>> >>
>>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>>> Blink
>>>>>>>> >> planner, not in Old planner.
>>>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>>>> e.g.
>>>>>>>> >> Dimension Table Join [1],
>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>> [4], and so on.
>>>>>>>> >>
>>>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>>>> It is
>>>>>>>> >> frustrating for users to set
>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>> it's
>>>>>>>> >> surprising to see unsupported
>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>> planner.
>>>>>>>> >>
>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>> feautures and
>>>>>>>> >> prototyping for users.
>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>> suggest to set
>>>>>>>> >> default planner
>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>> >>
>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>> configuration[5]. In
>>>>>>>> >> this way, the existing
>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>> >>
>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>> another topic
>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>> >>
>>>>>>>> >> What do you think?
>>>>>>>> >>
>>>>>>>> >> Best,
>>>>>>>> >> Jark
>>>>>>>> >>
>>>>>>>> >> [1]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>> >> [2]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>> >> [3]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>> >> [4]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>> >> [5]:
>>>>>>>> >>
>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best, Jingsong Lee
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards
>>>>>>
>>>>>> Jeff Zhang
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Benoît Paris
>>>>> Ingénieur Machine Learning Explicable
>>>>> Tél : +33 6 60 74 23 00
>>>>> http://benoit.paris
>>>>> http://explicable.ml
>>>>>
>>>>
>>
>> --
>>
>> Benchao Li
>> School of Electronics Engineering and Computer Science, Peking University
>> Tel:+86-15650713730
>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>
>>
--
Best regards!
Rui Li
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Rui Li <li...@gmail.com>.
+1. I think it improves user experience.
On Mon, Jan 6, 2020 at 10:18 AM Zhenghua Gao <do...@gmail.com> wrote:
> +1 for making blink planner as the default planner for SQL Client since we
> have made a huge improvement in 1.10.
>
> *Best Regards,*
> *Zhenghua Gao*
>
>
> On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
>
>> +1
>>
>> We have used blink planner since 1.9.0 release in our production
>> environment, and it behaves really impressive.
>>
>> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>>
>>> +1 to make blink planner as the default planner for SQL Client, hence we
>>> can give the blink planner a bit more exposure.
>>>
>>> Best, Hequn
>>>
>>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>>
>>>> Hi Benoît,
>>>>
>>>> Thanks for the reminder. I will look into the issue and hopefully we
>>>> can target it into 1.9.2 and 1.10.
>>>>
>>>> Cheers,
>>>> Jark
>>>>
>>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>>> benoit.paris@centraliens-lille.org> wrote:
>>>>
>>>>> > If anyone finds that blink planner has any significant defects and
>>>>> has a larger regression than the old planner, please let us know.
>>>>>
>>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>>> production are not covered:
>>>>> An edge case of Temporal Table Functions does not allow computed
>>>>> Tables (as opposed to TableSources) to be used on the query side in Blink (
>>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>>
>>>>> Cheers
>>>>> Ben
>>>>>
>>>>>
>>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>>
>>>>>> +1, I have already made blink as the default planner of flink
>>>>>> interpreter in Zeppelin
>>>>>>
>>>>>>
>>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>>
>>>>>>> Hi Jark,
>>>>>>>
>>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>>> I believe this new planner can be put into practice in production.
>>>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>>>> on.
>>>>>>>
>>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>>> be very grateful.
>>>>>>>
>>>>>>> Best,
>>>>>>> Jingsong Lee
>>>>>>>
>>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1 for this.
>>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>>> planner.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Leonard
>>>>>>>>
>>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>>> >
>>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>>> >
>>>>>>>> > Best,
>>>>>>>> > Terry Wang
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>>> >>
>>>>>>>> >> Hi everyone,
>>>>>>>> >>
>>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>>> improvements,
>>>>>>>> >> including:
>>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>>> >> - fully support all data types in Hive
>>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>>> >>
>>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>>> Blink
>>>>>>>> >> planner, not in Old planner.
>>>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>>>> e.g.
>>>>>>>> >> Dimension Table Join [1],
>>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization
>>>>>>>> [4], and so on.
>>>>>>>> >>
>>>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>>>> It is
>>>>>>>> >> frustrating for users to set
>>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>>> it's
>>>>>>>> >> surprising to see unsupported
>>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>>> planner.
>>>>>>>> >>
>>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>>> feautures and
>>>>>>>> >> prototyping for users.
>>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>>> suggest to set
>>>>>>>> >> default planner
>>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>>> >>
>>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>>> configuration[5]. In
>>>>>>>> >> this way, the existing
>>>>>>>> >> environment is still compatible and unaffected.
>>>>>>>> >>
>>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>>> another topic
>>>>>>>> >> and is out of scope of this discussion.
>>>>>>>> >>
>>>>>>>> >> What do you think?
>>>>>>>> >>
>>>>>>>> >> Best,
>>>>>>>> >> Jark
>>>>>>>> >>
>>>>>>>> >> [1]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>>> >> [2]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>>> >> [3]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>>> >> [4]:
>>>>>>>> >>
>>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>>> >> [5]:
>>>>>>>> >>
>>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best, Jingsong Lee
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards
>>>>>>
>>>>>> Jeff Zhang
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Benoît Paris
>>>>> Ingénieur Machine Learning Explicable
>>>>> Tél : +33 6 60 74 23 00
>>>>> http://benoit.paris
>>>>> http://explicable.ml
>>>>>
>>>>
>>
>> --
>>
>> Benchao Li
>> School of Electronics Engineering and Computer Science, Peking University
>> Tel:+86-15650713730
>> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>>
>>
--
Best regards!
Rui Li
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Zhenghua Gao <do...@gmail.com>.
+1 for making blink planner as the default planner for SQL Client since we
have made a huge improvement in 1.10.
*Best Regards,*
*Zhenghua Gao*
On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
> +1
>
> We have used blink planner since 1.9.0 release in our production
> environment, and it behaves really impressive.
>
> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>
>> +1 to make blink planner as the default planner for SQL Client, hence we
>> can give the blink planner a bit more exposure.
>>
>> Best, Hequn
>>
>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Benoît,
>>>
>>> Thanks for the reminder. I will look into the issue and hopefully we can
>>> target it into 1.9.2 and 1.10.
>>>
>>> Cheers,
>>> Jark
>>>
>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>> benoit.paris@centraliens-lille.org> wrote:
>>>
>>>> > If anyone finds that blink planner has any significant defects and
>>>> has a larger regression than the old planner, please let us know.
>>>>
>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>> production are not covered:
>>>> An edge case of Temporal Table Functions does not allow computed Tables
>>>> (as opposed to TableSources) to be used on the query side in Blink (
>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>
>>>> Cheers
>>>> Ben
>>>>
>>>>
>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>
>>>>> +1, I have already made blink as the default planner of flink
>>>>> interpreter in Zeppelin
>>>>>
>>>>>
>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>
>>>>>> Hi Jark,
>>>>>>
>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>> I believe this new planner can be put into practice in production.
>>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>>> on.
>>>>>>
>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>> be very grateful.
>>>>>>
>>>>>> Best,
>>>>>> Jingsong Lee
>>>>>>
>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>>
>>>>>>> +1 for this.
>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>> planner.
>>>>>>>
>>>>>>> Best,
>>>>>>> Leonard
>>>>>>>
>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>> >
>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>> >
>>>>>>> > Best,
>>>>>>> > Terry Wang
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>> >>
>>>>>>> >> Hi everyone,
>>>>>>> >>
>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>> improvements,
>>>>>>> >> including:
>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>> >> - fully support all data types in Hive
>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>> >>
>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>> Blink
>>>>>>> >> planner, not in Old planner.
>>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>>> e.g.
>>>>>>> >> Dimension Table Join [1],
>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>>>> and so on.
>>>>>>> >>
>>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>>> It is
>>>>>>> >> frustrating for users to set
>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>> it's
>>>>>>> >> surprising to see unsupported
>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>> planner.
>>>>>>> >>
>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>> feautures and
>>>>>>> >> prototyping for users.
>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>> suggest to set
>>>>>>> >> default planner
>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>> >>
>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>> configuration[5]. In
>>>>>>> >> this way, the existing
>>>>>>> >> environment is still compatible and unaffected.
>>>>>>> >>
>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>> another topic
>>>>>>> >> and is out of scope of this discussion.
>>>>>>> >>
>>>>>>> >> What do you think?
>>>>>>> >>
>>>>>>> >> Best,
>>>>>>> >> Jark
>>>>>>> >>
>>>>>>> >> [1]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>> >> [2]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>> >> [3]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>> >> [4]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>> >> [5]:
>>>>>>> >>
>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best, Jingsong Lee
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>>
>>>>> Jeff Zhang
>>>>>
>>>>
>>>>
>>>> --
>>>> Benoît Paris
>>>> Ingénieur Machine Learning Explicable
>>>> Tél : +33 6 60 74 23 00
>>>> http://benoit.paris
>>>> http://explicable.ml
>>>>
>>>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Zhenghua Gao <do...@gmail.com>.
+1 for making blink planner as the default planner for SQL Client since we
have made a huge improvement in 1.10.
*Best Regards,*
*Zhenghua Gao*
On Sun, Jan 5, 2020 at 2:42 PM Benchao Li <li...@gmail.com> wrote:
> +1
>
> We have used blink planner since 1.9.0 release in our production
> environment, and it behaves really impressive.
>
> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>
>> +1 to make blink planner as the default planner for SQL Client, hence we
>> can give the blink planner a bit more exposure.
>>
>> Best, Hequn
>>
>> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Benoît,
>>>
>>> Thanks for the reminder. I will look into the issue and hopefully we can
>>> target it into 1.9.2 and 1.10.
>>>
>>> Cheers,
>>> Jark
>>>
>>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>>> benoit.paris@centraliens-lille.org> wrote:
>>>
>>>> > If anyone finds that blink planner has any significant defects and
>>>> has a larger regression than the old planner, please let us know.
>>>>
>>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>>> production are not covered:
>>>> An edge case of Temporal Table Functions does not allow computed Tables
>>>> (as opposed to TableSources) to be used on the query side in Blink (
>>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>>
>>>> Cheers
>>>> Ben
>>>>
>>>>
>>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>>
>>>>> +1, I have already made blink as the default planner of flink
>>>>> interpreter in Zeppelin
>>>>>
>>>>>
>>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>>
>>>>>> Hi Jark,
>>>>>>
>>>>>> +1 for default blink planner in SQL-CLI.
>>>>>> I believe this new planner can be put into practice in production.
>>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>>> on.
>>>>>>
>>>>>> And I'd like to cc to user@flink.apache.org.
>>>>>> If anyone finds that blink planner has any significant defects and
>>>>>> has a larger regression than the old planner, please let us know. We will
>>>>>> be very grateful.
>>>>>>
>>>>>> Best,
>>>>>> Jingsong Lee
>>>>>>
>>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>>
>>>>>>> +1 for this.
>>>>>>> We bring many SQL/API features and enhance stability in 1.10
>>>>>>> release, and almost all of them happens in Blink planner.
>>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>>> users will have a better experience If we set Blink planner as default
>>>>>>> planner.
>>>>>>>
>>>>>>> Best,
>>>>>>> Leonard
>>>>>>>
>>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>>> >
>>>>>>> > Since what blink planner can do is a superset of flink planner,
>>>>>>> big +1 for changing the default planner to Blink planner from my side.
>>>>>>> >
>>>>>>> > Best,
>>>>>>> > Terry Wang
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>>> >>
>>>>>>> >> Hi everyone,
>>>>>>> >>
>>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>>> improvements,
>>>>>>> >> including:
>>>>>>> >> - support watermark statement and computed column in DDL
>>>>>>> >> - fully support all data types in Hive
>>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>>> >>
>>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>>> Blink
>>>>>>> >> planner, not in Old planner.
>>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>>> e.g.
>>>>>>> >> Dimension Table Join [1],
>>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>>>> and so on.
>>>>>>> >>
>>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>>> It is
>>>>>>> >> frustrating for users to set
>>>>>>> >> to blink planner manually when every time start a SQL CLI. And
>>>>>>> it's
>>>>>>> >> surprising to see unsupported
>>>>>>> >> exception if they trying out the new features but not switch
>>>>>>> planner.
>>>>>>> >>
>>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>>> feautures and
>>>>>>> >> prototyping for users.
>>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>>> suggest to set
>>>>>>> >> default planner
>>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>>> >>
>>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>>> configuration[5]. In
>>>>>>> >> this way, the existing
>>>>>>> >> environment is still compatible and unaffected.
>>>>>>> >>
>>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>>> another topic
>>>>>>> >> and is out of scope of this discussion.
>>>>>>> >>
>>>>>>> >> What do you think?
>>>>>>> >>
>>>>>>> >> Best,
>>>>>>> >> Jark
>>>>>>> >>
>>>>>>> >> [1]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>>> >> [2]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>>> >> [3]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>>> >> [4]:
>>>>>>> >>
>>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>>> >> [5]:
>>>>>>> >>
>>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best, Jingsong Lee
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>>
>>>>> Jeff Zhang
>>>>>
>>>>
>>>>
>>>> --
>>>> Benoît Paris
>>>> Ingénieur Machine Learning Explicable
>>>> Tél : +33 6 60 74 23 00
>>>> http://benoit.paris
>>>> http://explicable.ml
>>>>
>>>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by 贺小令 <go...@gmail.com>.
+1 for making the blink planner as default planner, the blink planner
becomes more stable since 1.10.
Benchao Li <li...@gmail.com> 于2020年1月5日周日 下午2:42写道：
> +1
>
> We have used blink planner since 1.9.0 release in our production
> environment, and it behaves really impressive.
>
> Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
>
> > +1 to make blink planner as the default planner for SQL Client, hence we
> > can give the blink planner a bit more exposure.
> >
> > Best, Hequn
> >
> > On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
> >
> >> Hi Benoît,
> >>
> >> Thanks for the reminder. I will look into the issue and hopefully we can
> >> target it into 1.9.2 and 1.10.
> >>
> >> Cheers,
> >> Jark
> >>
> >> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
> >> benoit.paris@centraliens-lille.org> wrote:
> >>
> >>> > If anyone finds that blink planner has any significant defects and
> >>> has a larger regression than the old planner, please let us know.
> >>>
> >>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
> >>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner
> in
> >>> production are not covered:
> >>> An edge case of Temporal Table Functions does not allow computed Tables
> >>> (as opposed to TableSources) to be used on the query side in Blink (
> >>> https://issues.apache.org/jira/browse/FLINK-14200)
> >>>
> >>> Cheers
> >>> Ben
> >>>
> >>>
> >>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
> >>>
> >>>> +1, I have already made blink as the default planner of flink
> >>>> interpreter in Zeppelin
> >>>>
> >>>>
> >>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
> >>>>
> >>>>> Hi Jark,
> >>>>>
> >>>>> +1 for default blink planner in SQL-CLI.
> >>>>> I believe this new planner can be put into practice in production.
> >>>>> We've worked hard for nearly a year, but the old planner didn't move
> >>>>> on.
> >>>>>
> >>>>> And I'd like to cc to user@flink.apache.org.
> >>>>> If anyone finds that blink planner has any significant defects and
> has
> >>>>> a larger regression than the old planner, please let us know. We
> will be
> >>>>> very grateful.
> >>>>>
> >>>>> Best,
> >>>>> Jingsong Lee
> >>>>>
> >>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
> >>>>>
> >>>>>> +1 for this.
> >>>>>> We bring many SQL/API features and enhance stability in 1.10
> release,
> >>>>>> and almost all of them happens in Blink planner.
> >>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
> >>>>>> users will have a better experience If we set Blink planner as
> default
> >>>>>> planner.
> >>>>>>
> >>>>>> Best,
> >>>>>> Leonard
> >>>>>>
> >>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
> >>>>>> >
> >>>>>> > Since what blink planner can do is a superset of flink planner,
> big
> >>>>>> +1 for changing the default planner to Blink planner from my side.
> >>>>>> >
> >>>>>> > Best,
> >>>>>> > Terry Wang
> >>>>>> >
> >>>>>> >
> >>>>>> >
> >>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
> >>>>>> >>
> >>>>>> >> Hi everyone,
> >>>>>> >>
> >>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
> >>>>>> improvements,
> >>>>>> >> including:
> >>>>>> >> - support watermark statement and computed column in DDL
> >>>>>> >> - fully support all data types in Hive
> >>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
> >>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
> >>>>>> >>
> >>>>>> >> However, all the features and improvements are only avaiable in
> >>>>>> Blink
> >>>>>> >> planner, not in Old planner.
> >>>>>> >> There are also some other features are limited in Blink planner,
> >>>>>> e.g.
> >>>>>> >> Dimension Table Join [1],
> >>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
> >>>>>> and so on.
> >>>>>> >>
> >>>>>> >> But Old planner is still the default planner in Table API & SQL.
> >>>>>> It is
> >>>>>> >> frustrating for users to set
> >>>>>> >> to blink planner manually when every time start a SQL CLI. And
> it's
> >>>>>> >> surprising to see unsupported
> >>>>>> >> exception if they trying out the new features but not switch
> >>>>>> planner.
> >>>>>> >>
> >>>>>> >> SQL CLI is a very important entrypoint for trying out new
> >>>>>> feautures and
> >>>>>> >> prototyping for users.
> >>>>>> >> In order to give new planner more exposures, I would like to
> >>>>>> suggest to set
> >>>>>> >> default planner
> >>>>>> >> for SQL Client to Blink planner before 1.10 release.
> >>>>>> >>
> >>>>>> >> The approach is just changing the default SQL CLI yaml
> >>>>>> configuration[5]. In
> >>>>>> >> this way, the existing
> >>>>>> >> environment is still compatible and unaffected.
> >>>>>> >>
> >>>>>> >> Changing the default planner for the whole Table API & SQL is
> >>>>>> another topic
> >>>>>> >> and is out of scope of this discussion.
> >>>>>> >>
> >>>>>> >> What do you think?
> >>>>>> >>
> >>>>>> >> Best,
> >>>>>> >> Jark
> >>>>>> >>
> >>>>>> >> [1]:
> >>>>>> >>
> >>>>>>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
> >>>>>> >> [2]:
> >>>>>> >>
> >>>>>>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
> >>>>>> >> [3]:
> >>>>>> >>
> >>>>>>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
> >>>>>> >> [4]:
> >>>>>> >>
> >>>>>>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
> >>>>>> >> [5]:
> >>>>>> >>
> >>>>>>
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
> >>>>>> >
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Best, Jingsong Lee
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>>
> >>> --
> >>> Benoît Paris
> >>> Ingénieur Machine Learning Explicable
> >>> Tél : +33 6 60 74 23 00
> >>> http://benoit.paris
> >>> http://explicable.ml
> >>>
> >>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Benchao Li <li...@gmail.com>.
+1
We have used blink planner since 1.9.0 release in our production
environment, and it behaves really impressive.
Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
> +1 to make blink planner as the default planner for SQL Client, hence we
> can give the blink planner a bit more exposure.
>
> Best, Hequn
>
> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Benoît,
>>
>> Thanks for the reminder. I will look into the issue and hopefully we can
>> target it into 1.9.2 and 1.10.
>>
>> Cheers,
>> Jark
>>
>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>> benoit.paris@centraliens-lille.org> wrote:
>>
>>> > If anyone finds that blink planner has any significant defects and
>>> has a larger regression than the old planner, please let us know.
>>>
>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>> production are not covered:
>>> An edge case of Temporal Table Functions does not allow computed Tables
>>> (as opposed to TableSources) to be used on the query side in Blink (
>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>
>>> Cheers
>>> Ben
>>>
>>>
>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>>> +1, I have already made blink as the default planner of flink
>>>> interpreter in Zeppelin
>>>>
>>>>
>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>
>>>>> Hi Jark,
>>>>>
>>>>> +1 for default blink planner in SQL-CLI.
>>>>> I believe this new planner can be put into practice in production.
>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>> on.
>>>>>
>>>>> And I'd like to cc to user@flink.apache.org.
>>>>> If anyone finds that blink planner has any significant defects and has
>>>>> a larger regression than the old planner, please let us know. We will be
>>>>> very grateful.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>
>>>>>> +1 for this.
>>>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>>>> and almost all of them happens in Blink planner.
>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>> users will have a better experience If we set Blink planner as default
>>>>>> planner.
>>>>>>
>>>>>> Best,
>>>>>> Leonard
>>>>>>
>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>> >
>>>>>> > Since what blink planner can do is a superset of flink planner, big
>>>>>> +1 for changing the default planner to Blink planner from my side.
>>>>>> >
>>>>>> > Best,
>>>>>> > Terry Wang
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>> >>
>>>>>> >> Hi everyone,
>>>>>> >>
>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>> improvements,
>>>>>> >> including:
>>>>>> >> - support watermark statement and computed column in DDL
>>>>>> >> - fully support all data types in Hive
>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>> >>
>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>> Blink
>>>>>> >> planner, not in Old planner.
>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>> e.g.
>>>>>> >> Dimension Table Join [1],
>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>>> and so on.
>>>>>> >>
>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>> It is
>>>>>> >> frustrating for users to set
>>>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>>>> >> surprising to see unsupported
>>>>>> >> exception if they trying out the new features but not switch
>>>>>> planner.
>>>>>> >>
>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>> feautures and
>>>>>> >> prototyping for users.
>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>> suggest to set
>>>>>> >> default planner
>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>> >>
>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>> configuration[5]. In
>>>>>> >> this way, the existing
>>>>>> >> environment is still compatible and unaffected.
>>>>>> >>
>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>> another topic
>>>>>> >> and is out of scope of this discussion.
>>>>>> >>
>>>>>> >> What do you think?
>>>>>> >>
>>>>>> >> Best,
>>>>>> >> Jark
>>>>>> >>
>>>>>> >> [1]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>> >> [2]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>> >> [3]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>> >> [4]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>> >> [5]:
>>>>>> >>
>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>> --
>>> Benoît Paris
>>> Ingénieur Machine Learning Explicable
>>> Tél : +33 6 60 74 23 00
>>> http://benoit.paris
>>> http://explicable.ml
>>>
>>
--
Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Benchao Li <li...@gmail.com>.
+1
We have used blink planner since 1.9.0 release in our production
environment, and it behaves really impressive.
Hequn Cheng <ch...@gmail.com> 于2020年1月5日周日 下午1:58写道：
> +1 to make blink planner as the default planner for SQL Client, hence we
> can give the blink planner a bit more exposure.
>
> Best, Hequn
>
> On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Benoît,
>>
>> Thanks for the reminder. I will look into the issue and hopefully we can
>> target it into 1.9.2 and 1.10.
>>
>> Cheers,
>> Jark
>>
>> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
>> benoit.paris@centraliens-lille.org> wrote:
>>
>>> > If anyone finds that blink planner has any significant defects and
>>> has a larger regression than the old planner, please let us know.
>>>
>>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>>> production are not covered:
>>> An edge case of Temporal Table Functions does not allow computed Tables
>>> (as opposed to TableSources) to be used on the query side in Blink (
>>> https://issues.apache.org/jira/browse/FLINK-14200)
>>>
>>> Cheers
>>> Ben
>>>
>>>
>>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>>> +1, I have already made blink as the default planner of flink
>>>> interpreter in Zeppelin
>>>>
>>>>
>>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>>
>>>>> Hi Jark,
>>>>>
>>>>> +1 for default blink planner in SQL-CLI.
>>>>> I believe this new planner can be put into practice in production.
>>>>> We've worked hard for nearly a year, but the old planner didn't move
>>>>> on.
>>>>>
>>>>> And I'd like to cc to user@flink.apache.org.
>>>>> If anyone finds that blink planner has any significant defects and has
>>>>> a larger regression than the old planner, please let us know. We will be
>>>>> very grateful.
>>>>>
>>>>> Best,
>>>>> Jingsong Lee
>>>>>
>>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>>
>>>>>> +1 for this.
>>>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>>>> and almost all of them happens in Blink planner.
>>>>>> SQL CLI is the most convenient entrypoint for me, I believe many
>>>>>> users will have a better experience If we set Blink planner as default
>>>>>> planner.
>>>>>>
>>>>>> Best,
>>>>>> Leonard
>>>>>>
>>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>>> >
>>>>>> > Since what blink planner can do is a superset of flink planner, big
>>>>>> +1 for changing the default planner to Blink planner from my side.
>>>>>> >
>>>>>> > Best,
>>>>>> > Terry Wang
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>>> >>
>>>>>> >> Hi everyone,
>>>>>> >>
>>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>>> improvements,
>>>>>> >> including:
>>>>>> >> - support watermark statement and computed column in DDL
>>>>>> >> - fully support all data types in Hive
>>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>>> >>
>>>>>> >> However, all the features and improvements are only avaiable in
>>>>>> Blink
>>>>>> >> planner, not in Old planner.
>>>>>> >> There are also some other features are limited in Blink planner,
>>>>>> e.g.
>>>>>> >> Dimension Table Join [1],
>>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>>> and so on.
>>>>>> >>
>>>>>> >> But Old planner is still the default planner in Table API & SQL.
>>>>>> It is
>>>>>> >> frustrating for users to set
>>>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>>>> >> surprising to see unsupported
>>>>>> >> exception if they trying out the new features but not switch
>>>>>> planner.
>>>>>> >>
>>>>>> >> SQL CLI is a very important entrypoint for trying out new
>>>>>> feautures and
>>>>>> >> prototyping for users.
>>>>>> >> In order to give new planner more exposures, I would like to
>>>>>> suggest to set
>>>>>> >> default planner
>>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>>> >>
>>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>>> configuration[5]. In
>>>>>> >> this way, the existing
>>>>>> >> environment is still compatible and unaffected.
>>>>>> >>
>>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>>> another topic
>>>>>> >> and is out of scope of this discussion.
>>>>>> >>
>>>>>> >> What do you think?
>>>>>> >>
>>>>>> >> Best,
>>>>>> >> Jark
>>>>>> >>
>>>>>> >> [1]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>>> >> [2]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>>> >> [3]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>>> >> [4]:
>>>>>> >>
>>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>>> >> [5]:
>>>>>> >>
>>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Best, Jingsong Lee
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>>
>>> --
>>> Benoît Paris
>>> Ingénieur Machine Learning Explicable
>>> Tél : +33 6 60 74 23 00
>>> http://benoit.paris
>>> http://explicable.ml
>>>
>>
--
Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Hequn Cheng <ch...@gmail.com>.
+1 to make blink planner as the default planner for SQL Client, hence we
can give the blink planner a bit more exposure.
Best, Hequn
On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
> Hi Benoît,
>
> Thanks for the reminder. I will look into the issue and hopefully we can
> target it into 1.9.2 and 1.10.
>
> Cheers,
> Jark
>
> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
> benoit.paris@centraliens-lille.org> wrote:
>
>> > If anyone finds that blink planner has any significant defects and
>> has a larger regression than the old planner, please let us know.
>>
>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>> production are not covered:
>> An edge case of Temporal Table Functions does not allow computed Tables
>> (as opposed to TableSources) to be used on the query side in Blink (
>> https://issues.apache.org/jira/browse/FLINK-14200)
>>
>> Cheers
>> Ben
>>
>>
>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>
>>> +1, I have already made blink as the default planner of flink
>>> interpreter in Zeppelin
>>>
>>>
>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>
>>>> Hi Jark,
>>>>
>>>> +1 for default blink planner in SQL-CLI.
>>>> I believe this new planner can be put into practice in production.
>>>> We've worked hard for nearly a year, but the old planner didn't move on.
>>>>
>>>> And I'd like to cc to user@flink.apache.org.
>>>> If anyone finds that blink planner has any significant defects and has
>>>> a larger regression than the old planner, please let us know. We will be
>>>> very grateful.
>>>>
>>>> Best,
>>>> Jingsong Lee
>>>>
>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>
>>>>> +1 for this.
>>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>>> and almost all of them happens in Blink planner.
>>>>> SQL CLI is the most convenient entrypoint for me, I believe many users
>>>>> will have a better experience If we set Blink planner as default planner.
>>>>>
>>>>> Best,
>>>>> Leonard
>>>>>
>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>> >
>>>>> > Since what blink planner can do is a superset of flink planner, big
>>>>> +1 for changing the default planner to Blink planner from my side.
>>>>> >
>>>>> > Best,
>>>>> > Terry Wang
>>>>> >
>>>>> >
>>>>> >
>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>> >>
>>>>> >> Hi everyone,
>>>>> >>
>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>> improvements,
>>>>> >> including:
>>>>> >> - support watermark statement and computed column in DDL
>>>>> >> - fully support all data types in Hive
>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>> >>
>>>>> >> However, all the features and improvements are only avaiable in
>>>>> Blink
>>>>> >> planner, not in Old planner.
>>>>> >> There are also some other features are limited in Blink planner,
>>>>> e.g.
>>>>> >> Dimension Table Join [1],
>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>> and so on.
>>>>> >>
>>>>> >> But Old planner is still the default planner in Table API & SQL. It
>>>>> is
>>>>> >> frustrating for users to set
>>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>>> >> surprising to see unsupported
>>>>> >> exception if they trying out the new features but not switch
>>>>> planner.
>>>>> >>
>>>>> >> SQL CLI is a very important entrypoint for trying out new feautures
>>>>> and
>>>>> >> prototyping for users.
>>>>> >> In order to give new planner more exposures, I would like to
>>>>> suggest to set
>>>>> >> default planner
>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>> >>
>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>> configuration[5]. In
>>>>> >> this way, the existing
>>>>> >> environment is still compatible and unaffected.
>>>>> >>
>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>> another topic
>>>>> >> and is out of scope of this discussion.
>>>>> >>
>>>>> >> What do you think?
>>>>> >>
>>>>> >> Best,
>>>>> >> Jark
>>>>> >>
>>>>> >> [1]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>> >> [2]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>> >> [3]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>> >> [4]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>> >> [5]:
>>>>> >>
>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>> >
>>>>>
>>>>>
>>>>
>>>> --
>>>> Best, Jingsong Lee
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>> --
>> Benoît Paris
>> Ingénieur Machine Learning Explicable
>> Tél : +33 6 60 74 23 00
>> http://benoit.paris
>> http://explicable.ml
>>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Hequn Cheng <ch...@gmail.com>.
+1 to make blink planner as the default planner for SQL Client, hence we
can give the blink planner a bit more exposure.
Best, Hequn
On Fri, Jan 3, 2020 at 6:32 PM Jark Wu <im...@gmail.com> wrote:
> Hi Benoît,
>
> Thanks for the reminder. I will look into the issue and hopefully we can
> target it into 1.9.2 and 1.10.
>
> Cheers,
> Jark
>
> On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
> benoit.paris@centraliens-lille.org> wrote:
>
>> > If anyone finds that blink planner has any significant defects and
>> has a larger regression than the old planner, please let us know.
>>
>> Overall, the Blink-exclusive features are must (TopN, deduplicate,
>> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
>> production are not covered:
>> An edge case of Temporal Table Functions does not allow computed Tables
>> (as opposed to TableSources) to be used on the query side in Blink (
>> https://issues.apache.org/jira/browse/FLINK-14200)
>>
>> Cheers
>> Ben
>>
>>
>> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>>
>>> +1, I have already made blink as the default planner of flink
>>> interpreter in Zeppelin
>>>
>>>
>>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>>
>>>> Hi Jark,
>>>>
>>>> +1 for default blink planner in SQL-CLI.
>>>> I believe this new planner can be put into practice in production.
>>>> We've worked hard for nearly a year, but the old planner didn't move on.
>>>>
>>>> And I'd like to cc to user@flink.apache.org.
>>>> If anyone finds that blink planner has any significant defects and has
>>>> a larger regression than the old planner, please let us know. We will be
>>>> very grateful.
>>>>
>>>> Best,
>>>> Jingsong Lee
>>>>
>>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>>
>>>>> +1 for this.
>>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>>> and almost all of them happens in Blink planner.
>>>>> SQL CLI is the most convenient entrypoint for me, I believe many users
>>>>> will have a better experience If we set Blink planner as default planner.
>>>>>
>>>>> Best,
>>>>> Leonard
>>>>>
>>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>>> >
>>>>> > Since what blink planner can do is a superset of flink planner, big
>>>>> +1 for changing the default planner to Blink planner from my side.
>>>>> >
>>>>> > Best,
>>>>> > Terry Wang
>>>>> >
>>>>> >
>>>>> >
>>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>> >>
>>>>> >> Hi everyone,
>>>>> >>
>>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>>> improvements,
>>>>> >> including:
>>>>> >> - support watermark statement and computed column in DDL
>>>>> >> - fully support all data types in Hive
>>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>>> >>
>>>>> >> However, all the features and improvements are only avaiable in
>>>>> Blink
>>>>> >> planner, not in Old planner.
>>>>> >> There are also some other features are limited in Blink planner,
>>>>> e.g.
>>>>> >> Dimension Table Join [1],
>>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>>> and so on.
>>>>> >>
>>>>> >> But Old planner is still the default planner in Table API & SQL. It
>>>>> is
>>>>> >> frustrating for users to set
>>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>>> >> surprising to see unsupported
>>>>> >> exception if they trying out the new features but not switch
>>>>> planner.
>>>>> >>
>>>>> >> SQL CLI is a very important entrypoint for trying out new feautures
>>>>> and
>>>>> >> prototyping for users.
>>>>> >> In order to give new planner more exposures, I would like to
>>>>> suggest to set
>>>>> >> default planner
>>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>>> >>
>>>>> >> The approach is just changing the default SQL CLI yaml
>>>>> configuration[5]. In
>>>>> >> this way, the existing
>>>>> >> environment is still compatible and unaffected.
>>>>> >>
>>>>> >> Changing the default planner for the whole Table API & SQL is
>>>>> another topic
>>>>> >> and is out of scope of this discussion.
>>>>> >>
>>>>> >> What do you think?
>>>>> >>
>>>>> >> Best,
>>>>> >> Jark
>>>>> >>
>>>>> >> [1]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>>> >> [2]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>>> >> [3]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>>> >> [4]:
>>>>> >>
>>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>>> >> [5]:
>>>>> >>
>>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>>> >
>>>>>
>>>>>
>>>>
>>>> --
>>>> Best, Jingsong Lee
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>> --
>> Benoît Paris
>> Ingénieur Machine Learning Explicable
>> Tél : +33 6 60 74 23 00
>> http://benoit.paris
>> http://explicable.ml
>>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jark Wu <im...@gmail.com>.
Hi Benoît,
Thanks for the reminder. I will look into the issue and hopefully we can
target it into 1.9.2 and 1.10.
Cheers,
Jark
On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
benoit.paris@centraliens-lille.org> wrote:
> > If anyone finds that blink planner has any significant defects and has
> a larger regression than the old planner, please let us know.
>
> Overall, the Blink-exclusive features are must (TopN, deduplicate,
> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
> production are not covered:
> An edge case of Temporal Table Functions does not allow computed Tables
> (as opposed to TableSources) to be used on the query side in Blink (
> https://issues.apache.org/jira/browse/FLINK-14200)
>
> Cheers
> Ben
>
>
> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> +1, I have already made blink as the default planner of flink interpreter
>> in Zeppelin
>>
>>
>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>
>>> Hi Jark,
>>>
>>> +1 for default blink planner in SQL-CLI.
>>> I believe this new planner can be put into practice in production.
>>> We've worked hard for nearly a year, but the old planner didn't move on.
>>>
>>> And I'd like to cc to user@flink.apache.org.
>>> If anyone finds that blink planner has any significant defects and has a
>>> larger regression than the old planner, please let us know. We will be very
>>> grateful.
>>>
>>> Best,
>>> Jingsong Lee
>>>
>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>
>>>> +1 for this.
>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>> and almost all of them happens in Blink planner.
>>>> SQL CLI is the most convenient entrypoint for me, I believe many users
>>>> will have a better experience If we set Blink planner as default planner.
>>>>
>>>> Best,
>>>> Leonard
>>>>
>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>> >
>>>> > Since what blink planner can do is a superset of flink planner, big
>>>> +1 for changing the default planner to Blink planner from my side.
>>>> >
>>>> > Best,
>>>> > Terry Wang
>>>> >
>>>> >
>>>> >
>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>> >>
>>>> >> Hi everyone,
>>>> >>
>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>> improvements,
>>>> >> including:
>>>> >> - support watermark statement and computed column in DDL
>>>> >> - fully support all data types in Hive
>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>> >>
>>>> >> However, all the features and improvements are only avaiable in Blink
>>>> >> planner, not in Old planner.
>>>> >> There are also some other features are limited in Blink planner, e.g.
>>>> >> Dimension Table Join [1],
>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>> and so on.
>>>> >>
>>>> >> But Old planner is still the default planner in Table API & SQL. It
>>>> is
>>>> >> frustrating for users to set
>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>> >> surprising to see unsupported
>>>> >> exception if they trying out the new features but not switch planner.
>>>> >>
>>>> >> SQL CLI is a very important entrypoint for trying out new feautures
>>>> and
>>>> >> prototyping for users.
>>>> >> In order to give new planner more exposures, I would like to suggest
>>>> to set
>>>> >> default planner
>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>> >>
>>>> >> The approach is just changing the default SQL CLI yaml
>>>> configuration[5]. In
>>>> >> this way, the existing
>>>> >> environment is still compatible and unaffected.
>>>> >>
>>>> >> Changing the default planner for the whole Table API & SQL is
>>>> another topic
>>>> >> and is out of scope of this discussion.
>>>> >>
>>>> >> What do you think?
>>>> >>
>>>> >> Best,
>>>> >> Jark
>>>> >>
>>>> >> [1]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>> >> [2]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>> >> [3]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>> >> [4]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>> >> [5]:
>>>> >>
>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>> >
>>>>
>>>>
>>>
>>> --
>>> Best, Jingsong Lee
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> Benoît Paris
> Ingénieur Machine Learning Explicable
> Tél : +33 6 60 74 23 00
> http://benoit.paris
> http://explicable.ml
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jark Wu <im...@gmail.com>.
Hi Benoît,
Thanks for the reminder. I will look into the issue and hopefully we can
target it into 1.9.2 and 1.10.
Cheers,
Jark
On Fri, 3 Jan 2020 at 18:21, Benoît Paris <
benoit.paris@centraliens-lille.org> wrote:
> > If anyone finds that blink planner has any significant defects and has
> a larger regression than the old planner, please let us know.
>
> Overall, the Blink-exclusive features are must (TopN, deduplicate,
> LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
> production are not covered:
> An edge case of Temporal Table Functions does not allow computed Tables
> (as opposed to TableSources) to be used on the query side in Blink (
> https://issues.apache.org/jira/browse/FLINK-14200)
>
> Cheers
> Ben
>
>
> On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> +1, I have already made blink as the default planner of flink interpreter
>> in Zeppelin
>>
>>
>> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>>
>>> Hi Jark,
>>>
>>> +1 for default blink planner in SQL-CLI.
>>> I believe this new planner can be put into practice in production.
>>> We've worked hard for nearly a year, but the old planner didn't move on.
>>>
>>> And I'd like to cc to user@flink.apache.org.
>>> If anyone finds that blink planner has any significant defects and has a
>>> larger regression than the old planner, please let us know. We will be very
>>> grateful.
>>>
>>> Best,
>>> Jingsong Lee
>>>
>>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>>
>>>> +1 for this.
>>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>>> and almost all of them happens in Blink planner.
>>>> SQL CLI is the most convenient entrypoint for me, I believe many users
>>>> will have a better experience If we set Blink planner as default planner.
>>>>
>>>> Best,
>>>> Leonard
>>>>
>>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>> >
>>>> > Since what blink planner can do is a superset of flink planner, big
>>>> +1 for changing the default planner to Blink planner from my side.
>>>> >
>>>> > Best,
>>>> > Terry Wang
>>>> >
>>>> >
>>>> >
>>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>> >>
>>>> >> Hi everyone,
>>>> >>
>>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>>> improvements,
>>>> >> including:
>>>> >> - support watermark statement and computed column in DDL
>>>> >> - fully support all data types in Hive
>>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>>> >>
>>>> >> However, all the features and improvements are only avaiable in Blink
>>>> >> planner, not in Old planner.
>>>> >> There are also some other features are limited in Blink planner, e.g.
>>>> >> Dimension Table Join [1],
>>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4],
>>>> and so on.
>>>> >>
>>>> >> But Old planner is still the default planner in Table API & SQL. It
>>>> is
>>>> >> frustrating for users to set
>>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>>> >> surprising to see unsupported
>>>> >> exception if they trying out the new features but not switch planner.
>>>> >>
>>>> >> SQL CLI is a very important entrypoint for trying out new feautures
>>>> and
>>>> >> prototyping for users.
>>>> >> In order to give new planner more exposures, I would like to suggest
>>>> to set
>>>> >> default planner
>>>> >> for SQL Client to Blink planner before 1.10 release.
>>>> >>
>>>> >> The approach is just changing the default SQL CLI yaml
>>>> configuration[5]. In
>>>> >> this way, the existing
>>>> >> environment is still compatible and unaffected.
>>>> >>
>>>> >> Changing the default planner for the whole Table API & SQL is
>>>> another topic
>>>> >> and is out of scope of this discussion.
>>>> >>
>>>> >> What do you think?
>>>> >>
>>>> >> Best,
>>>> >> Jark
>>>> >>
>>>> >> [1]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>> >> [2]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>> >> [3]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>> >> [4]:
>>>> >>
>>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>> >> [5]:
>>>> >>
>>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>> >
>>>>
>>>>
>>>
>>> --
>>> Best, Jingsong Lee
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> Benoît Paris
> Ingénieur Machine Learning Explicable
> Tél : +33 6 60 74 23 00
> http://benoit.paris
> http://explicable.ml
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Benoît Paris <be...@centraliens-lille.org>.
> If anyone finds that blink planner has any significant defects and has a
larger regression than the old planner, please let us know.
Overall, the Blink-exclusive features are must (TopN, deduplicate,
LAST_VALUE, plan reuse, etc)! But all use cases of the Legacy planner in
production are not covered:
An edge case of Temporal Table Functions does not allow computed Tables (as
opposed to TableSources) to be used on the query side in Blink (
https://issues.apache.org/jira/browse/FLINK-14200)
Cheers
Ben
On Fri, Jan 3, 2020 at 10:00 AM Jeff Zhang <zj...@gmail.com> wrote:
> +1, I have already made blink as the default planner of flink interpreter
> in Zeppelin
>
>
> Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
>
>> Hi Jark,
>>
>> +1 for default blink planner in SQL-CLI.
>> I believe this new planner can be put into practice in production.
>> We've worked hard for nearly a year, but the old planner didn't move on.
>>
>> And I'd like to cc to user@flink.apache.org.
>> If anyone finds that blink planner has any significant defects and has a
>> larger regression than the old planner, please let us know. We will be very
>> grateful.
>>
>> Best,
>> Jingsong Lee
>>
>> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>>
>>> +1 for this.
>>> We bring many SQL/API features and enhance stability in 1.10 release,
>>> and almost all of them happens in Blink planner.
>>> SQL CLI is the most convenient entrypoint for me, I believe many users
>>> will have a better experience If we set Blink planner as default planner.
>>>
>>> Best,
>>> Leonard
>>>
>>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>> >
>>> > Since what blink planner can do is a superset of flink planner, big +1
>>> for changing the default planner to Blink planner from my side.
>>> >
>>> > Best,
>>> > Terry Wang
>>> >
>>> >
>>> >
>>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>> >>
>>> >> Hi everyone,
>>> >>
>>> >> In 1.10 release, Flink SQL supports many awesome features and
>>> improvements,
>>> >> including:
>>> >> - support watermark statement and computed column in DDL
>>> >> - fully support all data types in Hive
>>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>> >> - support INSERT OVERWRITE and INSERT PARTITION
>>> >>
>>> >> However, all the features and improvements are only avaiable in Blink
>>> >> planner, not in Old planner.
>>> >> There are also some other features are limited in Blink planner, e.g.
>>> >> Dimension Table Join [1],
>>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
>>> so on.
>>> >>
>>> >> But Old planner is still the default planner in Table API & SQL. It is
>>> >> frustrating for users to set
>>> >> to blink planner manually when every time start a SQL CLI. And it's
>>> >> surprising to see unsupported
>>> >> exception if they trying out the new features but not switch planner.
>>> >>
>>> >> SQL CLI is a very important entrypoint for trying out new feautures
>>> and
>>> >> prototyping for users.
>>> >> In order to give new planner more exposures, I would like to suggest
>>> to set
>>> >> default planner
>>> >> for SQL Client to Blink planner before 1.10 release.
>>> >>
>>> >> The approach is just changing the default SQL CLI yaml
>>> configuration[5]. In
>>> >> this way, the existing
>>> >> environment is still compatible and unaffected.
>>> >>
>>> >> Changing the default planner for the whole Table API & SQL is another
>>> topic
>>> >> and is out of scope of this discussion.
>>> >>
>>> >> What do you think?
>>> >>
>>> >> Best,
>>> >> Jark
>>> >>
>>> >> [1]:
>>> >>
>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>> >> [2]:
>>> >>
>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>> >> [3]:
>>> >>
>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>> >> [4]:
>>> >>
>>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>> >> [5]:
>>> >>
>>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>> >
>>>
>>>
>>
>> --
>> Best, Jingsong Lee
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Benoît Paris
Ingénieur Machine Learning Explicable
Tél : +33 6 60 74 23 00
http://benoit.paris
http://explicable.ml
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jeff Zhang <zj...@gmail.com>.
+1, I have already made blink as the default planner of flink interpreter
in Zeppelin
Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
> Hi Jark,
>
> +1 for default blink planner in SQL-CLI.
> I believe this new planner can be put into practice in production.
> We've worked hard for nearly a year, but the old planner didn't move on.
>
> And I'd like to cc to user@flink.apache.org.
> If anyone finds that blink planner has any significant defects and has a
> larger regression than the old planner, please let us know. We will be very
> grateful.
>
> Best,
> Jingsong Lee
>
> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>
>> +1 for this.
>> We bring many SQL/API features and enhance stability in 1.10 release, and
>> almost all of them happens in Blink planner.
>> SQL CLI is the most convenient entrypoint for me, I believe many users
>> will have a better experience If we set Blink planner as default planner.
>>
>> Best,
>> Leonard
>>
>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>> >
>> > Since what blink planner can do is a superset of flink planner, big +1
>> for changing the default planner to Blink planner from my side.
>> >
>> > Best,
>> > Terry Wang
>> >
>> >
>> >
>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>> >>
>> >> Hi everyone,
>> >>
>> >> In 1.10 release, Flink SQL supports many awesome features and
>> improvements,
>> >> including:
>> >> - support watermark statement and computed column in DDL
>> >> - fully support all data types in Hive
>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>> >> - support INSERT OVERWRITE and INSERT PARTITION
>> >>
>> >> However, all the features and improvements are only avaiable in Blink
>> >> planner, not in Old planner.
>> >> There are also some other features are limited in Blink planner, e.g.
>> >> Dimension Table Join [1],
>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
>> so on.
>> >>
>> >> But Old planner is still the default planner in Table API & SQL. It is
>> >> frustrating for users to set
>> >> to blink planner manually when every time start a SQL CLI. And it's
>> >> surprising to see unsupported
>> >> exception if they trying out the new features but not switch planner.
>> >>
>> >> SQL CLI is a very important entrypoint for trying out new feautures and
>> >> prototyping for users.
>> >> In order to give new planner more exposures, I would like to suggest
>> to set
>> >> default planner
>> >> for SQL Client to Blink planner before 1.10 release.
>> >>
>> >> The approach is just changing the default SQL CLI yaml
>> configuration[5]. In
>> >> this way, the existing
>> >> environment is still compatible and unaffected.
>> >>
>> >> Changing the default planner for the whole Table API & SQL is another
>> topic
>> >> and is out of scope of this discussion.
>> >>
>> >> What do you think?
>> >>
>> >> Best,
>> >> Jark
>> >>
>> >> [1]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>> >> [2]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>> >> [3]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>> >> [4]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>> >> [5]:
>> >>
>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>> >
>>
>>
>
> --
> Best, Jingsong Lee
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jeff Zhang <zj...@gmail.com>.
+1, I have already made blink as the default planner of flink interpreter
in Zeppelin
Jingsong Li <ji...@gmail.com> 于2020年1月3日周五 下午4:37写道：
> Hi Jark,
>
> +1 for default blink planner in SQL-CLI.
> I believe this new planner can be put into practice in production.
> We've worked hard for nearly a year, but the old planner didn't move on.
>
> And I'd like to cc to user@flink.apache.org.
> If anyone finds that blink planner has any significant defects and has a
> larger regression than the old planner, please let us know. We will be very
> grateful.
>
> Best,
> Jingsong Lee
>
> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>
>> +1 for this.
>> We bring many SQL/API features and enhance stability in 1.10 release, and
>> almost all of them happens in Blink planner.
>> SQL CLI is the most convenient entrypoint for me, I believe many users
>> will have a better experience If we set Blink planner as default planner.
>>
>> Best,
>> Leonard
>>
>> > 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>> >
>> > Since what blink planner can do is a superset of flink planner, big +1
>> for changing the default planner to Blink planner from my side.
>> >
>> > Best,
>> > Terry Wang
>> >
>> >
>> >
>> >> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>> >>
>> >> Hi everyone,
>> >>
>> >> In 1.10 release, Flink SQL supports many awesome features and
>> improvements,
>> >> including:
>> >> - support watermark statement and computed column in DDL
>> >> - fully support all data types in Hive
>> >> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>> >> - support INSERT OVERWRITE and INSERT PARTITION
>> >>
>> >> However, all the features and improvements are only avaiable in Blink
>> >> planner, not in Old planner.
>> >> There are also some other features are limited in Blink planner, e.g.
>> >> Dimension Table Join [1],
>> >> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
>> so on.
>> >>
>> >> But Old planner is still the default planner in Table API & SQL. It is
>> >> frustrating for users to set
>> >> to blink planner manually when every time start a SQL CLI. And it's
>> >> surprising to see unsupported
>> >> exception if they trying out the new features but not switch planner.
>> >>
>> >> SQL CLI is a very important entrypoint for trying out new feautures and
>> >> prototyping for users.
>> >> In order to give new planner more exposures, I would like to suggest
>> to set
>> >> default planner
>> >> for SQL Client to Blink planner before 1.10 release.
>> >>
>> >> The approach is just changing the default SQL CLI yaml
>> configuration[5]. In
>> >> this way, the existing
>> >> environment is still compatible and unaffected.
>> >>
>> >> Changing the default planner for the whole Table API & SQL is another
>> topic
>> >> and is out of scope of this discussion.
>> >>
>> >> What do you think?
>> >>
>> >> Best,
>> >> Jark
>> >>
>> >> [1]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>> >> [2]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>> >> [3]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>> >> [4]:
>> >>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>> >> [5]:
>> >>
>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>> >
>>
>>
>
> --
> Best, Jingsong Lee
>
--
Best Regards
Jeff Zhang
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Jark Wu <im...@gmail.com>.
Hi Timo,
I have the same idea about the two step approach.
+1 to change default planner for Table API in 1.11.
Best,
Jark
On Fri, 3 Jan 2020 at 17:06, Timo Walther <tw...@apache.org> wrote:
> Hi Jark,
>
> +1 for making the Blink planner the default planner for the SQL Client.
>
> I think for the Table API, we should give the planner a bit more
> exposure and target the changing of a default planner for 1.11.
>
> What do you think about this two step approach?
>
> Regards,
> Timo
>
> On 03.01.20 09:37, Jingsong Li wrote:
> > Hi Jark,
> >
> > +1 for default blink planner in SQL-CLI.
> > I believe this new planner can be put into practice in production.
> > We've worked hard for nearly a year, but the old planner didn't move on.
> >
> > And I'd like to cc to user@flink.apache.org.
> > If anyone finds that blink planner has any significant defects and has a
> > larger regression than the old planner, please let us know. We will be
> very
> > grateful.
> >
> > Best,
> > Jingsong Lee
> >
> > On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
> >
> >> +1 for this.
> >> We bring many SQL/API features and enhance stability in 1.10 release,
> and
> >> almost all of them happens in Blink planner.
> >> SQL CLI is the most convenient entrypoint for me, I believe many users
> >> will have a better experience If we set Blink planner as default
> planner.
> >>
> >> Best,
> >> Leonard
> >>
> >>> 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
> >>>
> >>> Since what blink planner can do is a superset of flink planner, big +1
> >> for changing the default planner to Blink planner from my side.
> >>>
> >>> Best,
> >>> Terry Wang
> >>>
> >>>
> >>>
> >>>> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
> >>>>
> >>>> Hi everyone,
> >>>>
> >>>> In 1.10 release, Flink SQL supports many awesome features and
> >> improvements,
> >>>> including:
> >>>> - support watermark statement and computed column in DDL
> >>>> - fully support all data types in Hive
> >>>> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
> >>>> - support INSERT OVERWRITE and INSERT PARTITION
> >>>>
> >>>> However, all the features and improvements are only avaiable in Blink
> >>>> planner, not in Old planner.
> >>>> There are also some other features are limited in Blink planner, e.g.
> >>>> Dimension Table Join [1],
> >>>> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
> >> so on.
> >>>>
> >>>> But Old planner is still the default planner in Table API & SQL. It is
> >>>> frustrating for users to set
> >>>> to blink planner manually when every time start a SQL CLI. And it's
> >>>> surprising to see unsupported
> >>>> exception if they trying out the new features but not switch planner.
> >>>>
> >>>> SQL CLI is a very important entrypoint for trying out new feautures
> and
> >>>> prototyping for users.
> >>>> In order to give new planner more exposures, I would like to suggest
> to
> >> set
> >>>> default planner
> >>>> for SQL Client to Blink planner before 1.10 release.
> >>>>
> >>>> The approach is just changing the default SQL CLI yaml
> >> configuration[5]. In
> >>>> this way, the existing
> >>>> environment is still compatible and unaffected.
> >>>>
> >>>> Changing the default planner for the whole Table API & SQL is another
> >> topic
> >>>> and is out of scope of this discussion.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Best,
> >>>> Jark
> >>>>
> >>>> [1]:
> >>>>
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
> >>>> [2]:
> >>>>
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
> >>>> [3]:
> >>>>
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
> >>>> [4]:
> >>>>
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
> >>>> [5]:
> >>>>
> >>
> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
> >>>
> >>
> >>
> >
>
>
Re: [DISCUSS] Set default planner for SQL Client to Blink planner in
1.10 release
Posted by Timo Walther <tw...@apache.org>.
Hi Jark,
+1 for making the Blink planner the default planner for the SQL Client.
I think for the Table API, we should give the planner a bit more
exposure and target the changing of a default planner for 1.11.
What do you think about this two step approach?
Regards,
Timo
On 03.01.20 09:37, Jingsong Li wrote:
> Hi Jark,
>
> +1 for default blink planner in SQL-CLI.
> I believe this new planner can be put into practice in production.
> We've worked hard for nearly a year, but the old planner didn't move on.
>
> And I'd like to cc to user@flink.apache.org.
> If anyone finds that blink planner has any significant defects and has a
> larger regression than the old planner, please let us know. We will be very
> grateful.
>
> Best,
> Jingsong Lee
>
> On Fri, Jan 3, 2020 at 4:14 PM Leonard Xu <xb...@gmail.com> wrote:
>
>> +1 for this.
>> We bring many SQL/API features and enhance stability in 1.10 release, and
>> almost all of them happens in Blink planner.
>> SQL CLI is the most convenient entrypoint for me, I believe many users
>> will have a better experience If we set Blink planner as default planner.
>>
>> Best,
>> Leonard
>>
>>> 在 2020年1月3日，15:16，Terry Wang <zj...@gmail.com> 写道：
>>>
>>> Since what blink planner can do is a superset of flink planner, big +1
>> for changing the default planner to Blink planner from my side.
>>>
>>> Best,
>>> Terry Wang
>>>
>>>
>>>
>>>> 2020年1月3日 15:00，Jark Wu <im...@gmail.com> 写道：
>>>>
>>>> Hi everyone,
>>>>
>>>> In 1.10 release, Flink SQL supports many awesome features and
>> improvements,
>>>> including:
>>>> - support watermark statement and computed column in DDL
>>>> - fully support all data types in Hive
>>>> - Batch SQL performance improvements (TPC-DS 7x than Hive MR)
>>>> - support INSERT OVERWRITE and INSERT PARTITION
>>>>
>>>> However, all the features and improvements are only avaiable in Blink
>>>> planner, not in Old planner.
>>>> There are also some other features are limited in Blink planner, e.g.
>>>> Dimension Table Join [1],
>>>> TopN [2], Deduplicate [3], streaming aggregates optimization [4], and
>> so on.
>>>>
>>>> But Old planner is still the default planner in Table API & SQL. It is
>>>> frustrating for users to set
>>>> to blink planner manually when every time start a SQL CLI. And it's
>>>> surprising to see unsupported
>>>> exception if they trying out the new features but not switch planner.
>>>>
>>>> SQL CLI is a very important entrypoint for trying out new feautures and
>>>> prototyping for users.
>>>> In order to give new planner more exposures, I would like to suggest to
>> set
>>>> default planner
>>>> for SQL Client to Blink planner before 1.10 release.
>>>>
>>>> The approach is just changing the default SQL CLI yaml
>> configuration[5]. In
>>>> this way, the existing
>>>> environment is still compatible and unaffected.
>>>>
>>>> Changing the default planner for the whole Table API & SQL is another
>> topic
>>>> and is out of scope of this discussion.
>>>>
>>>> What do you think?
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> [1]:
>>>>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/streaming/joins.html#join-with-a-temporal-table
>>>> [2]:
>>>>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#top-n
>>>> [3]:
>>>>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#deduplication
>>>> [4]:
>>>>
>> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tuning/streaming_aggregation_optimization.html
>>>> [5]:
>>>>
>> https://github.com/apache/flink/blob/master/flink-table/flink-sql-client/conf/sql-client-defaults.yaml#L100
>>>
>>
>>
>