You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Forward Xu <fo...@gmail.com> on 2019/12/21 00:15:37 UTC
[DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Hi everybody,
I'd like to kick off a discussion on FLIP-90: Support SQL 2016-2017 JSON
functions in Flink SQL.
Implement Support SQL 2016-2017 JSON functions in Flink SQL[1].
Would love to hear your thoughts.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-90%3A+Support+SQL+2016-2017+JSON+functions+in+Flink+SQL
<https://docs.google.com/document/d/1JfaFYIFOAY8P2pFhOYNCQ9RTzwF4l85_bnTvImOLKMk/edit#heading=h.76mb88ca6yjp>
Best,
ForwardXu
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
hi，Dann，Thanks you very much.
Best,
Forward
Danny Chan <yu...@gmail.com> 于2020年1月8日周三 下午2:25写道：
> Thanks Forward ~
>
> I would help to view code from the Calcite side.
>
> Best,
> Danny Chan
> 在 2020年1月7日 +0800 PM2:01，Forward Xu <fo...@gmail.com>，写道：
> > Thanks Jark for checking the doc. hi,Timo, please help to check to see if
> > there is anything else to add.
> >
> > Best,
> > Forward
> >
> > Jark Wu <im...@gmail.com> 于2020年1月6日周一 下午2:58写道：
> >
> > > Thanks Forward for the updating. It is much more clearer now about the
> > > returning type, especially JSON_VALUE.
> > > The design doc looks good to me now.
> > >
> > > Best,
> > > Jark
> > >
> > > On Fri, 3 Jan 2020 at 21:42, Forward Xu <fo...@gmail.com>
> wrote:
> > >
> > > > Hi Timo, Jack,
> > > > Well, I added additional column to describe the return type of each
> > > > function and
> > > > updated the google doc.
> > > >
> > > > Best,
> > > > Forward
> > > >
> > > > Jark Wu <im...@gmail.com> 于2020年1月3日周五 下午5:48写道：
> > > >
> > > > > Hi Timo,
> > > > >
> > > > > That's a good point.
> > > > > We didn't introduce any new types. We will use the function
> definition
> > > > > defined by Calcite[1].
> > > > > So all the functions return STRING/BOOLEAN.
> > > > >
> > > > > Hi Forward,
> > > > > I think we may need an additional column to describe the return
> type of
> > > > > each function.
> > > > >
> > > > > Best,
> > > > > Jark
> > > > >
> > > > > [1]:
> > > > >
> > > > >
> > > >
> > >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> > > > >
> > > > > On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > sorry for jumping into the discussion so late. I had a quick
> look at
> > > > the
> > > > > > FLIP. It looks very nice and detailed. I have one question that I
> > > could
> > > > > > not find in the FLIP itself. Maybe it is hidden in the long
> > > discussion
> > > > > > thread.
> > > > > >
> > > > > > What are the return types of all functions? Do we introduce new
> types
> > > > > > with this FLIP? Also the RAW types should be avoided. Do all
> > > functions
> > > > > > return STRING/BOOLEAN?
> > > > > >
> > > > > > Thanks,
> > > > > > Timo
> > > > > >
> > > > > >
> > > > > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > > > > Thanks a lot for the update. +1 to start a vote.
> > > > > > >
> > > > > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <
> forwardxu315@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Jark, Hequn,
> > > > > > > >
> > > > > > > > I have updated the documentation.
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > > Forward
> > > > > > > >
> > > > > > > > Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> > > > > > > >
> > > > > > > > > Hi Jark, Hequn,
> > > > > > > > >
> > > > > > > > > Thank you very much, Introducing new `TableSymbol`s sounds
> like a
> > > > > good
> > > > > > > > > idea. +1 for the proposal.
> > > > > > > > >
> > > > > > > > > I think this idea is good, I will add this in the
> documentation.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best, Forward
> > > > > > > > >
> > > > > > > > > Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日
> 下午3:41写道：
> > > > > > > > >
> > > > > > > > > > Hi Jark,
> > > > > > > > > >
> > > > > > > > > > Introducing new `TableSymbol`s sounds like a good idea.
> +1 for
> > > the
> > > > > > > > > > proposal.
> > > > > > > > > > @ForwardXu what do you think? Would be great if the
> document can
> > > > be
> > > > > > > > > > updated
> > > > > > > > > > accordingly.
> > > > > > > > > >
> > > > > > > > > > Best, Hequn
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <
> imjark@gmail.com>
> > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Thanks for looking into the design Hequn. I agree it
> would be
> > > > great
> > > > > > to
> > > > > > > > > > > have a full story design.
> > > > > > > > > > >
> > > > > > > > > > > For the ON ERROR and ON EMPTY clause in Table API,
> some initial
> > > > > > > > > > > thoughts in my mind is that
> > > > > > > > > > > we can introduce some new `TableSymbol`s as the second
> > > parameter
> > > > of
> > > > > > > > json
> > > > > > > > > > > function, e.g. `JsonErrorStrategy`.
> > > > > > > > > > >
> > > > > > > > > > > For example,
> > > > > > > > > > >
> > > > > > > > > > > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > > > > > > > > > > == is equal to Table API ==>
> > > > > > > > > > > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Jark
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <
> > > chenghequn@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Jark & ForwardXu,
> > > > > > > > > > > >
> > > > > > > > > > > > The design doc looks very nice! Only some minor
> feedback from
> > > my
> > > > > > > > side.
> > > > > > > > > > > >
> > > > > > > > > > > > As calcite has already implemented the JSON
> functions, I would
> > > > > > > > suppose
> > > > > > > > > > > > the semantics and implementation are right for SQL.
> > > > > > > > > > > >
> > > > > > > > > > > > For TableAPI, I think the most important is to keep
> align with
> > > > the
> > > > > > > > > > > > SQL(which has also been mentioned by Jark in the
> previous
> > > > > > > > discussion).
> > > > > > > > > > Have
> > > > > > > > > > > > an equivalent feature set for all APIs and maintain
> it
> > > otherwise
> > > > > > > > > > confusion
> > > > > > > > > > > > increases especially when more and more functions
> are added.
> > > The
> > > > > > > > > > document
> > > > > > > > > > > > has documented how to support TableAPI. I think this
> is very
> > > > good!
> > > > > > > > And
> > > > > > > > > > it
> > > > > > > > > > > > would be better to also include ON ERROR or ON EMPTY
> for Table
> > > > > API.
> > > > > > > > We
> > > > > > > > > > can
> > > > > > > > > > > > implement these features step by step, but maybe we
> should
> > > > design
> > > > > > all
> > > > > > > > > > these
> > > > > > > > > > > > once for all to avoid API changes later. Meanwhile,
> these
> > > > features
> > > > > > > > are
> > > > > > > > > > also
> > > > > > > > > > > > commonly required by users.
> > > > > > > > > > > >
> > > > > > > > > > > > Would be great to also have your opinions!
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Hequn
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <
> imjark@gmail.com>
> > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Forward,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for creating the FLIP. +1 to start a vote.
> > > > > > > > > > > > >
> > > > > > > > > > > > > @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> > > > > > ykt836@gmail.com>
> > > > > > > > ,
> > > > > > > > > > > > > could you help to review the design doc too?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Jark
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Mon, 23 Dec 2019 at 10:10, tison <
> wander4096@gmail.com>
> > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > modified:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in
Flink SQL
Posted by Danny Chan <yu...@gmail.com>.
Thanks Forward ~
I would help to view code from the Calcite side.
Best,
Danny Chan
在 2020年1月7日 +0800 PM2:01，Forward Xu <fo...@gmail.com>，写道：
> Thanks Jark for checking the doc. hi,Timo, please help to check to see if
> there is anything else to add.
>
> Best,
> Forward
>
> Jark Wu <im...@gmail.com> 于2020年1月6日周一 下午2:58写道：
>
> > Thanks Forward for the updating. It is much more clearer now about the
> > returning type, especially JSON_VALUE.
> > The design doc looks good to me now.
> >
> > Best,
> > Jark
> >
> > On Fri, 3 Jan 2020 at 21:42, Forward Xu <fo...@gmail.com> wrote:
> >
> > > Hi Timo, Jack,
> > > Well, I added additional column to describe the return type of each
> > > function and
> > > updated the google doc.
> > >
> > > Best,
> > > Forward
> > >
> > > Jark Wu <im...@gmail.com> 于2020年1月3日周五 下午5:48写道：
> > >
> > > > Hi Timo,
> > > >
> > > > That's a good point.
> > > > We didn't introduce any new types. We will use the function definition
> > > > defined by Calcite[1].
> > > > So all the functions return STRING/BOOLEAN.
> > > >
> > > > Hi Forward,
> > > > I think we may need an additional column to describe the return type of
> > > > each function.
> > > >
> > > > Best,
> > > > Jark
> > > >
> > > > [1]:
> > > >
> > > >
> > >
> > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> > > >
> > > > On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > sorry for jumping into the discussion so late. I had a quick look at
> > > the
> > > > > FLIP. It looks very nice and detailed. I have one question that I
> > could
> > > > > not find in the FLIP itself. Maybe it is hidden in the long
> > discussion
> > > > > thread.
> > > > >
> > > > > What are the return types of all functions? Do we introduce new types
> > > > > with this FLIP? Also the RAW types should be avoided. Do all
> > functions
> > > > > return STRING/BOOLEAN?
> > > > >
> > > > > Thanks,
> > > > > Timo
> > > > >
> > > > >
> > > > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > > > Thanks a lot for the update. +1 to start a vote.
> > > > > >
> > > > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <forwardxu315@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi Jark, Hequn,
> > > > > > >
> > > > > > > I have updated the documentation.
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > Forward
> > > > > > >
> > > > > > > Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> > > > > > >
> > > > > > > > Hi Jark, Hequn,
> > > > > > > >
> > > > > > > > Thank you very much, Introducing new `TableSymbol`s sounds like a
> > > > good
> > > > > > > > idea. +1 for the proposal.
> > > > > > > >
> > > > > > > > I think this idea is good, I will add this in the documentation.
> > > > > > > >
> > > > > > > >
> > > > > > > > Best, Forward
> > > > > > > >
> > > > > > > > Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> > > > > > > >
> > > > > > > > > Hi Jark,
> > > > > > > > >
> > > > > > > > > Introducing new `TableSymbol`s sounds like a good idea. +1 for
> > the
> > > > > > > > > proposal.
> > > > > > > > > @ForwardXu what do you think? Would be great if the document can
> > > be
> > > > > > > > > updated
> > > > > > > > > accordingly.
> > > > > > > > >
> > > > > > > > > Best, Hequn
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com>
> > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for looking into the design Hequn. I agree it would be
> > > great
> > > > > to
> > > > > > > > > > have a full story design.
> > > > > > > > > >
> > > > > > > > > > For the ON ERROR and ON EMPTY clause in Table API, some initial
> > > > > > > > > > thoughts in my mind is that
> > > > > > > > > > we can introduce some new `TableSymbol`s as the second
> > parameter
> > > of
> > > > > > > json
> > > > > > > > > > function, e.g. `JsonErrorStrategy`.
> > > > > > > > > >
> > > > > > > > > > For example,
> > > > > > > > > >
> > > > > > > > > > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > > > > > > > > > == is equal to Table API ==>
> > > > > > > > > > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Jark
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <
> > chenghequn@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Jark & ForwardXu,
> > > > > > > > > > >
> > > > > > > > > > > The design doc looks very nice! Only some minor feedback from
> > my
> > > > > > > side.
> > > > > > > > > > >
> > > > > > > > > > > As calcite has already implemented the JSON functions, I would
> > > > > > > suppose
> > > > > > > > > > > the semantics and implementation are right for SQL.
> > > > > > > > > > >
> > > > > > > > > > > For TableAPI, I think the most important is to keep align with
> > > the
> > > > > > > > > > > SQL(which has also been mentioned by Jark in the previous
> > > > > > > discussion).
> > > > > > > > > Have
> > > > > > > > > > > an equivalent feature set for all APIs and maintain it
> > otherwise
> > > > > > > > > confusion
> > > > > > > > > > > increases especially when more and more functions are added.
> > The
> > > > > > > > > document
> > > > > > > > > > > has documented how to support TableAPI. I think this is very
> > > good!
> > > > > > > And
> > > > > > > > > it
> > > > > > > > > > > would be better to also include ON ERROR or ON EMPTY for Table
> > > > API.
> > > > > > > We
> > > > > > > > > can
> > > > > > > > > > > implement these features step by step, but maybe we should
> > > design
> > > > > all
> > > > > > > > > these
> > > > > > > > > > > once for all to avoid API changes later. Meanwhile, these
> > > features
> > > > > > > are
> > > > > > > > > also
> > > > > > > > > > > commonly required by users.
> > > > > > > > > > >
> > > > > > > > > > > Would be great to also have your opinions!
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Hequn
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com>
> > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Forward,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for creating the FLIP. +1 to start a vote.
> > > > > > > > > > > >
> > > > > > > > > > > > @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> > > > > ykt836@gmail.com>
> > > > > > > ,
> > > > > > > > > > > > could you help to review the design doc too?
> > > > > > > > > > > >
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Jark
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com>
> > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > modified:
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> > https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
Thanks Jark for checking the doc. hi,Timo, please help to check to see if
there is anything else to add.
Best,
Forward
Jark Wu <im...@gmail.com> 于2020年1月6日周一 下午2:58写道：
> Thanks Forward for the updating. It is much more clearer now about the
> returning type, especially JSON_VALUE.
> The design doc looks good to me now.
>
> Best,
> Jark
>
> On Fri, 3 Jan 2020 at 21:42, Forward Xu <fo...@gmail.com> wrote:
>
> > Hi Timo, Jack,
> > Well, I added additional column to describe the return type of each
> > function and
> > updated the google doc.
> >
> > Best,
> > Forward
> >
> > Jark Wu <im...@gmail.com> 于2020年1月3日周五 下午5:48写道：
> >
> > > Hi Timo,
> > >
> > > That's a good point.
> > > We didn't introduce any new types. We will use the function definition
> > > defined by Calcite[1].
> > > So all the functions return STRING/BOOLEAN.
> > >
> > > Hi Forward,
> > > I think we may need an additional column to describe the return type of
> > > each function.
> > >
> > > Best,
> > > Jark
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> > >
> > > On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org> wrote:
> > >
> > > > Hi,
> > > >
> > > > sorry for jumping into the discussion so late. I had a quick look at
> > the
> > > > FLIP. It looks very nice and detailed. I have one question that I
> could
> > > > not find in the FLIP itself. Maybe it is hidden in the long
> discussion
> > > > thread.
> > > >
> > > > What are the return types of all functions? Do we introduce new types
> > > > with this FLIP? Also the RAW types should be avoided. Do all
> functions
> > > > return STRING/BOOLEAN?
> > > >
> > > > Thanks,
> > > > Timo
> > > >
> > > >
> > > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > > Thanks a lot for the update. +1 to start a vote.
> > > > >
> > > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <forwardxu315@gmail.com
> >
> > > > wrote:
> > > > >
> > > > >> Hi Jark, Hequn,
> > > > >>
> > > > >> I have updated the documentation.
> > > > >>
> > > > >> Best,
> > > > >>
> > > > >> Forward
> > > > >>
> > > > >> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> > > > >>
> > > > >>> Hi Jark, Hequn,
> > > > >>>
> > > > >>> Thank you very much, Introducing new `TableSymbol`s sounds like a
> > > good
> > > > >>> idea. +1 for the proposal.
> > > > >>>
> > > > >>> I think this idea is good, I will add this in the documentation.
> > > > >>>
> > > > >>>
> > > > >>> Best, Forward
> > > > >>>
> > > > >>> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> > > > >>>
> > > > >>>> Hi Jark,
> > > > >>>>
> > > > >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for
> the
> > > > >>>> proposal.
> > > > >>>> @ForwardXu what do you think? Would be great if the document can
> > be
> > > > >>>> updated
> > > > >>>> accordingly.
> > > > >>>>
> > > > >>>> Best, Hequn
> > > > >>>>
> > > > >>>>
> > > > >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com>
> wrote:
> > > > >>>>
> > > > >>>>> Thanks for looking into the design Hequn. I agree it would be
> > great
> > > > to
> > > > >>>>> have a full story design.
> > > > >>>>>
> > > > >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> > > > >>>>> thoughts in my mind is that
> > > > >>>>> we can introduce some new `TableSymbol`s as the second
> parameter
> > of
> > > > >> json
> > > > >>>>> function, e.g. `JsonErrorStrategy`.
> > > > >>>>>
> > > > >>>>> For example,
> > > > >>>>>
> > > > >>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > > > >>>>> == is equal to Table API ==>
> > > > >>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > > > >>>>>
> > > > >>>>> Best,
> > > > >>>>> Jark
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <
> chenghequn@gmail.com>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Hi Jark & ForwardXu,
> > > > >>>>>>
> > > > >>>>>> The design doc looks very nice! Only some minor feedback from
> my
> > > > >> side.
> > > > >>>>>>
> > > > >>>>>> As calcite has already implemented the JSON functions, I would
> > > > >> suppose
> > > > >>>>>> the semantics and implementation are right for SQL.
> > > > >>>>>>
> > > > >>>>>> For TableAPI, I think the most important is to keep align with
> > the
> > > > >>>>>> SQL(which has also been mentioned by Jark in the previous
> > > > >> discussion).
> > > > >>>> Have
> > > > >>>>>> an equivalent feature set for all APIs and maintain it
> otherwise
> > > > >>>> confusion
> > > > >>>>>> increases especially when more and more functions are added.
> The
> > > > >>>> document
> > > > >>>>>> has documented how to support TableAPI. I think this is very
> > good!
> > > > >> And
> > > > >>>> it
> > > > >>>>>> would be better to also include ON ERROR or ON EMPTY for Table
> > > API.
> > > > >> We
> > > > >>>> can
> > > > >>>>>> implement these features step by step, but maybe we should
> > design
> > > > all
> > > > >>>> these
> > > > >>>>>> once for all to avoid API changes later. Meanwhile, these
> > features
> > > > >> are
> > > > >>>> also
> > > > >>>>>> commonly required by users.
> > > > >>>>>>
> > > > >>>>>> Would be great to also have your opinions!
> > > > >>>>>>
> > > > >>>>>> Best,
> > > > >>>>>> Hequn
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com>
> > > wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Forward,
> > > > >>>>>>>
> > > > >>>>>>> Thanks for creating the FLIP. +1 to start a vote.
> > > > >>>>>>>
> > > > >>>>>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> > > > ykt836@gmail.com>
> > > > >> ,
> > > > >>>>>>> could you help to review the design doc too?
> > > > >>>>>>>
> > > > >>>>>>> Best,
> > > > >>>>>>> Jark
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com>
> > > wrote:
> > > > >>>>>>>
> > > > >>>>>>>> modified:
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>
> > > > >>
> > > >
> > >
> >
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > > >
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Jark Wu <im...@gmail.com>.
Thanks Forward for the updating. It is much more clearer now about the
returning type, especially JSON_VALUE.
The design doc looks good to me now.
Best,
Jark
On Fri, 3 Jan 2020 at 21:42, Forward Xu <fo...@gmail.com> wrote:
> Hi Timo, Jack,
> Well, I added additional column to describe the return type of each
> function and
> updated the google doc.
>
> Best,
> Forward
>
> Jark Wu <im...@gmail.com> 于2020年1月3日周五 下午5:48写道：
>
> > Hi Timo,
> >
> > That's a good point.
> > We didn't introduce any new types. We will use the function definition
> > defined by Calcite[1].
> > So all the functions return STRING/BOOLEAN.
> >
> > Hi Forward,
> > I think we may need an additional column to describe the return type of
> > each function.
> >
> > Best,
> > Jark
> >
> > [1]:
> >
> >
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
> >
> > On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org> wrote:
> >
> > > Hi,
> > >
> > > sorry for jumping into the discussion so late. I had a quick look at
> the
> > > FLIP. It looks very nice and detailed. I have one question that I could
> > > not find in the FLIP itself. Maybe it is hidden in the long discussion
> > > thread.
> > >
> > > What are the return types of all functions? Do we introduce new types
> > > with this FLIP? Also the RAW types should be avoided. Do all functions
> > > return STRING/BOOLEAN?
> > >
> > > Thanks,
> > > Timo
> > >
> > >
> > > On 31.12.19 09:39, Hequn Cheng wrote:
> > > > Thanks a lot for the update. +1 to start a vote.
> > > >
> > > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <fo...@gmail.com>
> > > wrote:
> > > >
> > > >> Hi Jark, Hequn,
> > > >>
> > > >> I have updated the documentation.
> > > >>
> > > >> Best,
> > > >>
> > > >> Forward
> > > >>
> > > >> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> > > >>
> > > >>> Hi Jark, Hequn,
> > > >>>
> > > >>> Thank you very much, Introducing new `TableSymbol`s sounds like a
> > good
> > > >>> idea. +1 for the proposal.
> > > >>>
> > > >>> I think this idea is good, I will add this in the documentation.
> > > >>>
> > > >>>
> > > >>> Best, Forward
> > > >>>
> > > >>> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> > > >>>
> > > >>>> Hi Jark,
> > > >>>>
> > > >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> > > >>>> proposal.
> > > >>>> @ForwardXu what do you think? Would be great if the document can
> be
> > > >>>> updated
> > > >>>> accordingly.
> > > >>>>
> > > >>>> Best, Hequn
> > > >>>>
> > > >>>>
> > > >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
> > > >>>>
> > > >>>>> Thanks for looking into the design Hequn. I agree it would be
> great
> > > to
> > > >>>>> have a full story design.
> > > >>>>>
> > > >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> > > >>>>> thoughts in my mind is that
> > > >>>>> we can introduce some new `TableSymbol`s as the second parameter
> of
> > > >> json
> > > >>>>> function, e.g. `JsonErrorStrategy`.
> > > >>>>>
> > > >>>>> For example,
> > > >>>>>
> > > >>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > > >>>>> == is equal to Table API ==>
> > > >>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > > >>>>>
> > > >>>>> Best,
> > > >>>>> Jark
> > > >>>>>
> > > >>>>>
> > > >>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com>
> > > >> wrote:
> > > >>>>>
> > > >>>>>> Hi Jark & ForwardXu,
> > > >>>>>>
> > > >>>>>> The design doc looks very nice! Only some minor feedback from my
> > > >> side.
> > > >>>>>>
> > > >>>>>> As calcite has already implemented the JSON functions, I would
> > > >> suppose
> > > >>>>>> the semantics and implementation are right for SQL.
> > > >>>>>>
> > > >>>>>> For TableAPI, I think the most important is to keep align with
> the
> > > >>>>>> SQL(which has also been mentioned by Jark in the previous
> > > >> discussion).
> > > >>>> Have
> > > >>>>>> an equivalent feature set for all APIs and maintain it otherwise
> > > >>>> confusion
> > > >>>>>> increases especially when more and more functions are added. The
> > > >>>> document
> > > >>>>>> has documented how to support TableAPI. I think this is very
> good!
> > > >> And
> > > >>>> it
> > > >>>>>> would be better to also include ON ERROR or ON EMPTY for Table
> > API.
> > > >> We
> > > >>>> can
> > > >>>>>> implement these features step by step, but maybe we should
> design
> > > all
> > > >>>> these
> > > >>>>>> once for all to avoid API changes later. Meanwhile, these
> features
> > > >> are
> > > >>>> also
> > > >>>>>> commonly required by users.
> > > >>>>>>
> > > >>>>>> Would be great to also have your opinions!
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Hequn
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com>
> > wrote:
> > > >>>>>>
> > > >>>>>>> Hi Forward,
> > > >>>>>>>
> > > >>>>>>> Thanks for creating the FLIP. +1 to start a vote.
> > > >>>>>>>
> > > >>>>>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> > > ykt836@gmail.com>
> > > >> ,
> > > >>>>>>> could you help to review the design doc too?
> > > >>>>>>>
> > > >>>>>>> Best,
> > > >>>>>>> Jark
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com>
> > wrote:
> > > >>>>>>>
> > > >>>>>>>> modified:
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>
> > > >>
> > >
> >
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > >
> > >
> >
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
Hi Timo, Jack,
Well, I added additional column to describe the return type of each
function and
updated the google doc.
Best,
Forward
Jark Wu <im...@gmail.com> 于2020年1月3日周五 下午5:48写道：
> Hi Timo,
>
> That's a good point.
> We didn't introduce any new types. We will use the function definition
> defined by Calcite[1].
> So all the functions return STRING/BOOLEAN.
>
> Hi Forward,
> I think we may need an additional column to describe the return type of
> each function.
>
> Best,
> Jark
>
> [1]:
>
> https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
>
> On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org> wrote:
>
> > Hi,
> >
> > sorry for jumping into the discussion so late. I had a quick look at the
> > FLIP. It looks very nice and detailed. I have one question that I could
> > not find in the FLIP itself. Maybe it is hidden in the long discussion
> > thread.
> >
> > What are the return types of all functions? Do we introduce new types
> > with this FLIP? Also the RAW types should be avoided. Do all functions
> > return STRING/BOOLEAN?
> >
> > Thanks,
> > Timo
> >
> >
> > On 31.12.19 09:39, Hequn Cheng wrote:
> > > Thanks a lot for the update. +1 to start a vote.
> > >
> > > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <fo...@gmail.com>
> > wrote:
> > >
> > >> Hi Jark, Hequn,
> > >>
> > >> I have updated the documentation.
> > >>
> > >> Best,
> > >>
> > >> Forward
> > >>
> > >> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> > >>
> > >>> Hi Jark, Hequn,
> > >>>
> > >>> Thank you very much, Introducing new `TableSymbol`s sounds like a
> good
> > >>> idea. +1 for the proposal.
> > >>>
> > >>> I think this idea is good, I will add this in the documentation.
> > >>>
> > >>>
> > >>> Best, Forward
> > >>>
> > >>> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> > >>>
> > >>>> Hi Jark,
> > >>>>
> > >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> > >>>> proposal.
> > >>>> @ForwardXu what do you think? Would be great if the document can be
> > >>>> updated
> > >>>> accordingly.
> > >>>>
> > >>>> Best, Hequn
> > >>>>
> > >>>>
> > >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
> > >>>>
> > >>>>> Thanks for looking into the design Hequn. I agree it would be great
> > to
> > >>>>> have a full story design.
> > >>>>>
> > >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> > >>>>> thoughts in my mind is that
> > >>>>> we can introduce some new `TableSymbol`s as the second parameter of
> > >> json
> > >>>>> function, e.g. `JsonErrorStrategy`.
> > >>>>>
> > >>>>> For example,
> > >>>>>
> > >>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > >>>>> == is equal to Table API ==>
> > >>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> > >>>>>
> > >>>>> Best,
> > >>>>> Jark
> > >>>>>
> > >>>>>
> > >>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com>
> > >> wrote:
> > >>>>>
> > >>>>>> Hi Jark & ForwardXu,
> > >>>>>>
> > >>>>>> The design doc looks very nice! Only some minor feedback from my
> > >> side.
> > >>>>>>
> > >>>>>> As calcite has already implemented the JSON functions, I would
> > >> suppose
> > >>>>>> the semantics and implementation are right for SQL.
> > >>>>>>
> > >>>>>> For TableAPI, I think the most important is to keep align with the
> > >>>>>> SQL(which has also been mentioned by Jark in the previous
> > >> discussion).
> > >>>> Have
> > >>>>>> an equivalent feature set for all APIs and maintain it otherwise
> > >>>> confusion
> > >>>>>> increases especially when more and more functions are added. The
> > >>>> document
> > >>>>>> has documented how to support TableAPI. I think this is very good!
> > >> And
> > >>>> it
> > >>>>>> would be better to also include ON ERROR or ON EMPTY for Table
> API.
> > >> We
> > >>>> can
> > >>>>>> implement these features step by step, but maybe we should design
> > all
> > >>>> these
> > >>>>>> once for all to avoid API changes later. Meanwhile, these features
> > >> are
> > >>>> also
> > >>>>>> commonly required by users.
> > >>>>>>
> > >>>>>> Would be great to also have your opinions!
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Hequn
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com>
> wrote:
> > >>>>>>
> > >>>>>>> Hi Forward,
> > >>>>>>>
> > >>>>>>> Thanks for creating the FLIP. +1 to start a vote.
> > >>>>>>>
> > >>>>>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> > ykt836@gmail.com>
> > >> ,
> > >>>>>>> could you help to review the design doc too?
> > >>>>>>>
> > >>>>>>> Best,
> > >>>>>>> Jark
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com>
> wrote:
> > >>>>>>>
> > >>>>>>>> modified:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>
> > >>
> >
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> > >>>>>>>>
> > >>>>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
> >
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Jark Wu <im...@gmail.com>.
Hi Timo,
That's a good point.
We didn't introduce any new types. We will use the function definition
defined by Calcite[1].
So all the functions return STRING/BOOLEAN.
Hi Forward,
I think we may need an additional column to describe the return type of
each function.
Best,
Jark
[1]:
https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/fun/SqlJsonQueryFunction.java
On Fri, 3 Jan 2020 at 17:30, Timo Walther <tw...@apache.org> wrote:
> Hi,
>
> sorry for jumping into the discussion so late. I had a quick look at the
> FLIP. It looks very nice and detailed. I have one question that I could
> not find in the FLIP itself. Maybe it is hidden in the long discussion
> thread.
>
> What are the return types of all functions? Do we introduce new types
> with this FLIP? Also the RAW types should be avoided. Do all functions
> return STRING/BOOLEAN?
>
> Thanks,
> Timo
>
>
> On 31.12.19 09:39, Hequn Cheng wrote:
> > Thanks a lot for the update. +1 to start a vote.
> >
> > On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <fo...@gmail.com>
> wrote:
> >
> >> Hi Jark, Hequn,
> >>
> >> I have updated the documentation.
> >>
> >> Best,
> >>
> >> Forward
> >>
> >> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> >>
> >>> Hi Jark, Hequn,
> >>>
> >>> Thank you very much, Introducing new `TableSymbol`s sounds like a good
> >>> idea. +1 for the proposal.
> >>>
> >>> I think this idea is good, I will add this in the documentation.
> >>>
> >>>
> >>> Best, Forward
> >>>
> >>> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> >>>
> >>>> Hi Jark,
> >>>>
> >>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> >>>> proposal.
> >>>> @ForwardXu what do you think? Would be great if the document can be
> >>>> updated
> >>>> accordingly.
> >>>>
> >>>> Best, Hequn
> >>>>
> >>>>
> >>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
> >>>>
> >>>>> Thanks for looking into the design Hequn. I agree it would be great
> to
> >>>>> have a full story design.
> >>>>>
> >>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
> >>>>> thoughts in my mind is that
> >>>>> we can introduce some new `TableSymbol`s as the second parameter of
> >> json
> >>>>> function, e.g. `JsonErrorStrategy`.
> >>>>>
> >>>>> For example,
> >>>>>
> >>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> >>>>> == is equal to Table API ==>
> >>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> >>>>>
> >>>>> Best,
> >>>>> Jark
> >>>>>
> >>>>>
> >>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> Hi Jark & ForwardXu,
> >>>>>>
> >>>>>> The design doc looks very nice! Only some minor feedback from my
> >> side.
> >>>>>>
> >>>>>> As calcite has already implemented the JSON functions, I would
> >> suppose
> >>>>>> the semantics and implementation are right for SQL.
> >>>>>>
> >>>>>> For TableAPI, I think the most important is to keep align with the
> >>>>>> SQL(which has also been mentioned by Jark in the previous
> >> discussion).
> >>>> Have
> >>>>>> an equivalent feature set for all APIs and maintain it otherwise
> >>>> confusion
> >>>>>> increases especially when more and more functions are added. The
> >>>> document
> >>>>>> has documented how to support TableAPI. I think this is very good!
> >> And
> >>>> it
> >>>>>> would be better to also include ON ERROR or ON EMPTY for Table API.
> >> We
> >>>> can
> >>>>>> implement these features step by step, but maybe we should design
> all
> >>>> these
> >>>>>> once for all to avoid API changes later. Meanwhile, these features
> >> are
> >>>> also
> >>>>>> commonly required by users.
> >>>>>>
> >>>>>> Would be great to also have your opinions!
> >>>>>>
> >>>>>> Best,
> >>>>>> Hequn
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hi Forward,
> >>>>>>>
> >>>>>>> Thanks for creating the FLIP. +1 to start a vote.
> >>>>>>>
> >>>>>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <
> ykt836@gmail.com>
> >> ,
> >>>>>>> could you help to review the design doc too?
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Jark
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> modified:
> >>>>>>>>
> >>>>>>>>
> >>>>
> >>
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> >>>>>>>>
> >>>>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink
SQL
Posted by Timo Walther <tw...@apache.org>.
Hi,
sorry for jumping into the discussion so late. I had a quick look at the
FLIP. It looks very nice and detailed. I have one question that I could
not find in the FLIP itself. Maybe it is hidden in the long discussion
thread.
What are the return types of all functions? Do we introduce new types
with this FLIP? Also the RAW types should be avoided. Do all functions
return STRING/BOOLEAN?
Thanks,
Timo
On 31.12.19 09:39, Hequn Cheng wrote:
> Thanks a lot for the update. +1 to start a vote.
>
> On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <fo...@gmail.com> wrote:
>
>> Hi Jark, Hequn,
>>
>> I have updated the documentation.
>>
>> Best,
>>
>> Forward
>>
>> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
>>
>>> Hi Jark, Hequn,
>>>
>>> Thank you very much, Introducing new `TableSymbol`s sounds like a good
>>> idea. +1 for the proposal.
>>>
>>> I think this idea is good, I will add this in the documentation.
>>>
>>>
>>> Best, Forward
>>>
>>> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
>>>
>>>> Hi Jark,
>>>>
>>>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
>>>> proposal.
>>>> @ForwardXu what do you think? Would be great if the document can be
>>>> updated
>>>> accordingly.
>>>>
>>>> Best, Hequn
>>>>
>>>>
>>>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
>>>>
>>>>> Thanks for looking into the design Hequn. I agree it would be great to
>>>>> have a full story design.
>>>>>
>>>>> For the ON ERROR and ON EMPTY clause in Table API, some initial
>>>>> thoughts in my mind is that
>>>>> we can introduce some new `TableSymbol`s as the second parameter of
>> json
>>>>> function, e.g. `JsonErrorStrategy`.
>>>>>
>>>>> For example,
>>>>>
>>>>> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
>>>>> == is equal to Table API ==>
>>>>> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
>>>>>
>>>>> Best,
>>>>> Jark
>>>>>
>>>>>
>>>>> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com>
>> wrote:
>>>>>
>>>>>> Hi Jark & ForwardXu,
>>>>>>
>>>>>> The design doc looks very nice! Only some minor feedback from my
>> side.
>>>>>>
>>>>>> As calcite has already implemented the JSON functions, I would
>> suppose
>>>>>> the semantics and implementation are right for SQL.
>>>>>>
>>>>>> For TableAPI, I think the most important is to keep align with the
>>>>>> SQL(which has also been mentioned by Jark in the previous
>> discussion).
>>>> Have
>>>>>> an equivalent feature set for all APIs and maintain it otherwise
>>>> confusion
>>>>>> increases especially when more and more functions are added. The
>>>> document
>>>>>> has documented how to support TableAPI. I think this is very good!
>> And
>>>> it
>>>>>> would be better to also include ON ERROR or ON EMPTY for Table API.
>> We
>>>> can
>>>>>> implement these features step by step, but maybe we should design all
>>>> these
>>>>>> once for all to avoid API changes later. Meanwhile, these features
>> are
>>>> also
>>>>>> commonly required by users.
>>>>>>
>>>>>> Would be great to also have your opinions!
>>>>>>
>>>>>> Best,
>>>>>> Hequn
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Forward,
>>>>>>>
>>>>>>> Thanks for creating the FLIP. +1 to start a vote.
>>>>>>>
>>>>>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com>
>> ,
>>>>>>> could you help to review the design doc too?
>>>>>>>
>>>>>>> Best,
>>>>>>> Jark
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
>>>>>>>
>>>>>>>> modified:
>>>>>>>>
>>>>>>>>
>>>>
>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>>>>>>>>
>>>>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Hequn Cheng <ch...@gmail.com>.
Thanks a lot for the update. +1 to start a vote.
On Tue, Dec 31, 2019 at 2:38 PM Forward Xu <fo...@gmail.com> wrote:
> Hi Jark, Hequn,
>
> I have updated the documentation.
>
> Best,
>
> Forward
>
> Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
>
> > Hi Jark, Hequn,
> >
> > Thank you very much, Introducing new `TableSymbol`s sounds like a good
> > idea. +1 for the proposal.
> >
> > I think this idea is good, I will add this in the documentation.
> >
> >
> > Best, Forward
> >
> > Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> >
> >> Hi Jark,
> >>
> >> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> >> proposal.
> >> @ForwardXu what do you think? Would be great if the document can be
> >> updated
> >> accordingly.
> >>
> >> Best, Hequn
> >>
> >>
> >> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
> >>
> >> > Thanks for looking into the design Hequn. I agree it would be great to
> >> > have a full story design.
> >> >
> >> > For the ON ERROR and ON EMPTY clause in Table API, some initial
> >> > thoughts in my mind is that
> >> > we can introduce some new `TableSymbol`s as the second parameter of
> json
> >> > function, e.g. `JsonErrorStrategy`.
> >> >
> >> > For example,
> >> >
> >> > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> >> > == is equal to Table API ==>
> >> > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> >> >
> >> > Best,
> >> > Jark
> >> >
> >> >
> >> > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com>
> wrote:
> >> >
> >> >> Hi Jark & ForwardXu,
> >> >>
> >> >> The design doc looks very nice! Only some minor feedback from my
> side.
> >> >>
> >> >> As calcite has already implemented the JSON functions, I would
> suppose
> >> >> the semantics and implementation are right for SQL.
> >> >>
> >> >> For TableAPI, I think the most important is to keep align with the
> >> >> SQL(which has also been mentioned by Jark in the previous
> discussion).
> >> Have
> >> >> an equivalent feature set for all APIs and maintain it otherwise
> >> confusion
> >> >> increases especially when more and more functions are added. The
> >> document
> >> >> has documented how to support TableAPI. I think this is very good!
> And
> >> it
> >> >> would be better to also include ON ERROR or ON EMPTY for Table API.
> We
> >> can
> >> >> implement these features step by step, but maybe we should design all
> >> these
> >> >> once for all to avoid API changes later. Meanwhile, these features
> are
> >> also
> >> >> commonly required by users.
> >> >>
> >> >> Would be great to also have your opinions!
> >> >>
> >> >> Best,
> >> >> Hequn
> >> >>
> >> >>
> >> >> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
> >> >>
> >> >>> Hi Forward,
> >> >>>
> >> >>> Thanks for creating the FLIP. +1 to start a vote.
> >> >>>
> >> >>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com>
> ,
> >> >>> could you help to review the design doc too?
> >> >>>
> >> >>> Best,
> >> >>> Jark
> >> >>>
> >> >>>
> >> >>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
> >> >>>
> >> >>>> modified:
> >> >>>>
> >> >>>>
> >>
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> >> >>>>
> >> >>>
> >>
> >
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
Hi Jark, Hequn,
I have updated the documentation.
Best,
Forward
Forward Xu <fo...@gmail.com> 于2019年12月29日周日 下午4:01写道：
> Hi Jark, Hequn,
>
> Thank you very much, Introducing new `TableSymbol`s sounds like a good
> idea. +1 for the proposal.
>
> I think this idea is good, I will add this in the documentation.
>
>
> Best, Forward
>
> Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
>
>> Hi Jark,
>>
>> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
>> proposal.
>> @ForwardXu what do you think? Would be great if the document can be
>> updated
>> accordingly.
>>
>> Best, Hequn
>>
>>
>> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
>>
>> > Thanks for looking into the design Hequn. I agree it would be great to
>> > have a full story design.
>> >
>> > For the ON ERROR and ON EMPTY clause in Table API, some initial
>> > thoughts in my mind is that
>> > we can introduce some new `TableSymbol`s as the second parameter of json
>> > function, e.g. `JsonErrorStrategy`.
>> >
>> > For example,
>> >
>> > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
>> > == is equal to Table API ==>
>> > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
>> >
>> > Best,
>> > Jark
>> >
>> >
>> > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com> wrote:
>> >
>> >> Hi Jark & ForwardXu,
>> >>
>> >> The design doc looks very nice! Only some minor feedback from my side.
>> >>
>> >> As calcite has already implemented the JSON functions, I would suppose
>> >> the semantics and implementation are right for SQL.
>> >>
>> >> For TableAPI, I think the most important is to keep align with the
>> >> SQL(which has also been mentioned by Jark in the previous discussion).
>> Have
>> >> an equivalent feature set for all APIs and maintain it otherwise
>> confusion
>> >> increases especially when more and more functions are added. The
>> document
>> >> has documented how to support TableAPI. I think this is very good! And
>> it
>> >> would be better to also include ON ERROR or ON EMPTY for Table API. We
>> can
>> >> implement these features step by step, but maybe we should design all
>> these
>> >> once for all to avoid API changes later. Meanwhile, these features are
>> also
>> >> commonly required by users.
>> >>
>> >> Would be great to also have your opinions!
>> >>
>> >> Best,
>> >> Hequn
>> >>
>> >>
>> >> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
>> >>
>> >>> Hi Forward,
>> >>>
>> >>> Thanks for creating the FLIP. +1 to start a vote.
>> >>>
>> >>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> ,
>> >>> could you help to review the design doc too?
>> >>>
>> >>> Best,
>> >>> Jark
>> >>>
>> >>>
>> >>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
>> >>>
>> >>>> modified:
>> >>>>
>> >>>>
>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>> >>>>
>> >>>
>>
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
Hi Jark, Hequn,
Thank you very much, Introducing new `TableSymbol`s sounds like a good
idea. +1 for the proposal.
I think this idea is good, I will add this in the documentation.
Best, Forward
Hequn Cheng <ch...@gmail.com> 于2019年12月29日周日 下午3:41写道：
> Hi Jark,
>
> Introducing new `TableSymbol`s sounds like a good idea. +1 for the
> proposal.
> @ForwardXu what do you think? Would be great if the document can be updated
> accordingly.
>
> Best, Hequn
>
>
> On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
>
> > Thanks for looking into the design Hequn. I agree it would be great to
> > have a full story design.
> >
> > For the ON ERROR and ON EMPTY clause in Table API, some initial
> > thoughts in my mind is that
> > we can introduce some new `TableSymbol`s as the second parameter of json
> > function, e.g. `JsonErrorStrategy`.
> >
> > For example,
> >
> > JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> > == is equal to Table API ==>
> > v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
> >
> > Best,
> > Jark
> >
> >
> > On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com> wrote:
> >
> >> Hi Jark & ForwardXu,
> >>
> >> The design doc looks very nice! Only some minor feedback from my side.
> >>
> >> As calcite has already implemented the JSON functions, I would suppose
> >> the semantics and implementation are right for SQL.
> >>
> >> For TableAPI, I think the most important is to keep align with the
> >> SQL(which has also been mentioned by Jark in the previous discussion).
> Have
> >> an equivalent feature set for all APIs and maintain it otherwise
> confusion
> >> increases especially when more and more functions are added. The
> document
> >> has documented how to support TableAPI. I think this is very good! And
> it
> >> would be better to also include ON ERROR or ON EMPTY for Table API. We
> can
> >> implement these features step by step, but maybe we should design all
> these
> >> once for all to avoid API changes later. Meanwhile, these features are
> also
> >> commonly required by users.
> >>
> >> Would be great to also have your opinions!
> >>
> >> Best,
> >> Hequn
> >>
> >>
> >> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
> >>
> >>> Hi Forward,
> >>>
> >>> Thanks for creating the FLIP. +1 to start a vote.
> >>>
> >>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> ,
> >>> could you help to review the design doc too?
> >>>
> >>> Best,
> >>> Jark
> >>>
> >>>
> >>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
> >>>
> >>>> modified:
> >>>>
> >>>>
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
> >>>>
> >>>
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Hequn Cheng <ch...@gmail.com>.
Hi Jark,
Introducing new `TableSymbol`s sounds like a good idea. +1 for the proposal.
@ForwardXu what do you think? Would be great if the document can be updated
accordingly.
Best, Hequn
On Fri, Dec 27, 2019 at 4:14 PM Jark Wu <im...@gmail.com> wrote:
> Thanks for looking into the design Hequn. I agree it would be great to
> have a full story design.
>
> For the ON ERROR and ON EMPTY clause in Table API, some initial
> thoughts in my mind is that
> we can introduce some new `TableSymbol`s as the second parameter of json
> function, e.g. `JsonErrorStrategy`.
>
> For example,
>
> JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
> == is equal to Table API ==>
> v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
>
> Best,
> Jark
>
>
> On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com> wrote:
>
>> Hi Jark & ForwardXu,
>>
>> The design doc looks very nice! Only some minor feedback from my side.
>>
>> As calcite has already implemented the JSON functions, I would suppose
>> the semantics and implementation are right for SQL.
>>
>> For TableAPI, I think the most important is to keep align with the
>> SQL(which has also been mentioned by Jark in the previous discussion). Have
>> an equivalent feature set for all APIs and maintain it otherwise confusion
>> increases especially when more and more functions are added. The document
>> has documented how to support TableAPI. I think this is very good! And it
>> would be better to also include ON ERROR or ON EMPTY for Table API. We can
>> implement these features step by step, but maybe we should design all these
>> once for all to avoid API changes later. Meanwhile, these features are also
>> commonly required by users.
>>
>> Would be great to also have your opinions!
>>
>> Best,
>> Hequn
>>
>>
>> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
>>
>>> Hi Forward,
>>>
>>> Thanks for creating the FLIP. +1 to start a vote.
>>>
>>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> ,
>>> could you help to review the design doc too?
>>>
>>> Best,
>>> Jark
>>>
>>>
>>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
>>>
>>>> modified:
>>>>
>>>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>>>>
>>>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Jark Wu <im...@gmail.com>.
Thanks for looking into the design Hequn. I agree it would be great to have
a full story design.
For the ON ERROR and ON EMPTY clause in Table API, some initial thoughts in
my mind is that
we can introduce some new `TableSymbol`s as the second parameter of json
function, e.g. `JsonErrorStrategy`.
For example,
JSON_VALUE(v, 'lax $.b' ERROR ON ERROR)
== is equal to Table API ==>
v.jsonValue("lax $.b", JsonErrorStrategy.ERROR)
Best,
Jark
On Thu, 26 Dec 2019 at 23:37, Hequn Cheng <ch...@gmail.com> wrote:
> Hi Jark & ForwardXu,
>
> The design doc looks very nice! Only some minor feedback from my side.
>
> As calcite has already implemented the JSON functions, I would suppose the
> semantics and implementation are right for SQL.
>
> For TableAPI, I think the most important is to keep align with the
> SQL(which has also been mentioned by Jark in the previous discussion). Have
> an equivalent feature set for all APIs and maintain it otherwise confusion
> increases especially when more and more functions are added. The document
> has documented how to support TableAPI. I think this is very good! And it
> would be better to also include ON ERROR or ON EMPTY for Table API. We can
> implement these features step by step, but maybe we should design all these
> once for all to avoid API changes later. Meanwhile, these features are also
> commonly required by users.
>
> Would be great to also have your opinions!
>
> Best,
> Hequn
>
>
> On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
>
>> Hi Forward,
>>
>> Thanks for creating the FLIP. +1 to start a vote.
>>
>> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> ,
>> could you help to review the design doc too?
>>
>> Best,
>> Jark
>>
>>
>> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
>>
>>> modified:
>>>
>>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>>>
>>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Hequn Cheng <ch...@gmail.com>.
Hi Jark & ForwardXu,
The design doc looks very nice! Only some minor feedback from my side.
As calcite has already implemented the JSON functions, I would suppose the
semantics and implementation are right for SQL.
For TableAPI, I think the most important is to keep align with the
SQL(which has also been mentioned by Jark in the previous discussion). Have
an equivalent feature set for all APIs and maintain it otherwise confusion
increases especially when more and more functions are added. The document
has documented how to support TableAPI. I think this is very good! And it
would be better to also include ON ERROR or ON EMPTY for Table API. We can
implement these features step by step, but maybe we should design all these
once for all to avoid API changes later. Meanwhile, these features are also
commonly required by users.
Would be great to also have your opinions!
Best,
Hequn
On Mon, Dec 23, 2019 at 10:15 AM Jark Wu <im...@gmail.com> wrote:
> Hi Forward,
>
> Thanks for creating the FLIP. +1 to start a vote.
>
> @Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> ,
> could you help to review the design doc too?
>
> Best,
> Jark
>
>
> On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
>
>> modified:
>>
>> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>>
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Jark Wu <im...@gmail.com>.
Hi Forward,
Thanks for creating the FLIP. +1 to start a vote.
@Hequn Cheng <ch...@gmail.com> @Kurt Young <yk...@gmail.com> , could
you help to review the design doc too?
Best,
Jark
On Mon, 23 Dec 2019 at 10:10, tison <wa...@gmail.com> wrote:
> modified:
>
> https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
>
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by tison <wa...@gmail.com>.
modified:
https://lists.apache.org/x/thread.html/b3c0265cc2b660fe11ce550b84a831a7606de12908ff7ff0959a4794@%3Cdev.flink.apache.org%3E
Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by tison <wa...@gmail.com>.
FYI the previous discussion is here[1].
<https://lists.apache.org/x/thread.html/e259aa70432e4003e1598e8f8db844813a869a0dd96accfc1b73deb6@%3Cdev.flink.apache.org%3E>
Best,
tison.
[1]
https://lists.apache.org/x/thread.html/e259aa70432e4003e1598e8f8db844813a869a0dd96accfc1b73deb6@%3Cdev.flink.apache.org%3E
Forward Xu <fo...@gmail.com> 于2019年12月21日周六 上午8:20写道：
> Hi everybody,
>
>
> I'd like to kick off a discussion on FLIP-90: Support SQL 2016-2017 JSON
> functions in Flink SQL.
>
>
> Implement Support SQL 2016-2017 JSON functions in Flink SQL[1].
>
>
>
> Would love to hear your thoughts.
>
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-90%3A+Support+SQL+2016-2017+JSON+functions+in+Flink+SQL
>
>
> Best,
>
> ForwardXu
>
> >
>
[DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL
Posted by Forward Xu <fo...@gmail.com>.
Hi everybody,
I'd like to kick off a discussion on FLIP-90: Support SQL 2016-2017 JSON
functions in Flink SQL.
Implement Support SQL 2016-2017 JSON functions in Flink SQL[1].
Would love to hear your thoughts.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-90%3A+Support+SQL+2016-2017+JSON+functions+in+Flink+SQL
Best,
ForwardXu
>