You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Wei Zhong <we...@gmail.com> on 2020/03/09 14:09:52 UTC
[DISCUSS] FLIP-114: Support Python UDF in SQL Client
Hi everyone,
I would like to start discussion about how to support Python UDF in SQL Client.
Flink Python UDF(FLIP-58[1]) has already been introduced in the release of 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
SQL Client defines UDF via the environment file and has its own CLI implementation to manage dependencies, but neither of which supports Python UDF. We want to introduce the support of Python UDF for SQL Client, including the registration and the dependency management of Python UDF.
Here is the design doc:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
Looking forward to your feedback!
Best,
Wei
[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by Hequn Cheng <he...@apache.org>.
Hi Wei,
Thanks a lot for the proposal! +1 for the VOTE.
Best,
Hequn
On Mon, Mar 30, 2020 at 3:31 PM Dian Fu <di...@gmail.com> wrote:
> Thanks Wei for this work! +1 to bring up the VOTE thread.
>
> > 在 2020年3月30日，下午2:43，jincheng sun <su...@gmail.com> 写道：
> >
> > Hi Wei,
> >
> > +1, Thanks for this discussion which is crucial for SQL users to use
> > PyFlink. Would be great to bring up the VOTE thread.
> >
> > Best,
> > Jincheng
> >
> >
> > Wei Zhong <we...@gmail.com> 于2020年3月30日周一 下午2:38写道：
> >
> >> Hi everyone,
> >>
> >> Are there more comments about this FLIP? If not, I would like to bring
> up
> >> the VOTE.
> >>
> >> Best,
> >> Wei
> >>
> >>> 在 2020年3月9日，23:18，Xingbo Huang <hx...@gmail.com> 写道：
> >>>
> >>> Hi Godfrey,
> >>> thanks for your suggestion.
> >>> I have added two examples how to use python UDF
> >>> in SQL and how to start sql-client.sh with full python dependencies In
> >> FLIP.
> >>>
> >>> Best,
> >>> Xingo
> >>>
> >>> godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
> >>>
> >>>> Hi Wei, thanks for the proposal.
> >>>>
> >>>> I think it's better to give two more examples, one is how to use
> python
> >> UDF
> >>>> in SQL, another is how to start sql-client.sh with full python
> >>>> dependencies.
> >>>>
> >>>> Best,
> >>>> Godfrey
> >>>>
> >>>> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
> >>>>
> >>>>> Hi everyone,
> >>>>>
> >>>>> I would like to start discussion about how to support Python UDF in
> SQL
> >>>>> Client.
> >>>>>
> >>>>> Flink Python UDF(FLIP-58[1]) has already been introduced in the
> release
> >>>> of
> >>>>> 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
> >>>>>
> >>>>> SQL Client defines UDF via the environment file and has its own CLI
> >>>>> implementation to manage dependencies, but neither of which supports
> >>>> Python
> >>>>> UDF. We want to introduce the support of Python UDF for SQL Client,
> >>>>> including the registration and the dependency management of Python
> UDF.
> >>>>>
> >>>>> Here is the design doc:
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
> >>>>>
> >>>>> Looking forward to your feedback!
> >>>>>
> >>>>> Best,
> >>>>> Wei
> >>>>>
> >>>>> [1]
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
> >>>>> [2]
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by Dian Fu <di...@gmail.com>.
Thanks Wei for this work! +1 to bring up the VOTE thread.
> 在 2020年3月30日，下午2:43，jincheng sun <su...@gmail.com> 写道：
>
> Hi Wei,
>
> +1, Thanks for this discussion which is crucial for SQL users to use
> PyFlink. Would be great to bring up the VOTE thread.
>
> Best,
> Jincheng
>
>
> Wei Zhong <we...@gmail.com> 于2020年3月30日周一 下午2:38写道：
>
>> Hi everyone,
>>
>> Are there more comments about this FLIP? If not, I would like to bring up
>> the VOTE.
>>
>> Best,
>> Wei
>>
>>> 在 2020年3月9日，23:18，Xingbo Huang <hx...@gmail.com> 写道：
>>>
>>> Hi Godfrey,
>>> thanks for your suggestion.
>>> I have added two examples how to use python UDF
>>> in SQL and how to start sql-client.sh with full python dependencies In
>> FLIP.
>>>
>>> Best,
>>> Xingo
>>>
>>> godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
>>>
>>>> Hi Wei, thanks for the proposal.
>>>>
>>>> I think it's better to give two more examples, one is how to use python
>> UDF
>>>> in SQL, another is how to start sql-client.sh with full python
>>>> dependencies.
>>>>
>>>> Best,
>>>> Godfrey
>>>>
>>>> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I would like to start discussion about how to support Python UDF in SQL
>>>>> Client.
>>>>>
>>>>> Flink Python UDF(FLIP-58[1]) has already been introduced in the release
>>>> of
>>>>> 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
>>>>>
>>>>> SQL Client defines UDF via the environment file and has its own CLI
>>>>> implementation to manage dependencies, but neither of which supports
>>>> Python
>>>>> UDF. We want to introduce the support of Python UDF for SQL Client,
>>>>> including the registration and the dependency management of Python UDF.
>>>>>
>>>>> Here is the design doc:
>>>>>
>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
>>>>>
>>>>> Looking forward to your feedback!
>>>>>
>>>>> Best,
>>>>> Wei
>>>>>
>>>>> [1]
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
>>>>> [2]
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
>>>>>
>>>>>
>>>>
>>
>>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by jincheng sun <su...@gmail.com>.
Hi Wei,
+1, Thanks for this discussion which is crucial for SQL users to use
PyFlink. Would be great to bring up the VOTE thread.
Best,
Jincheng
Wei Zhong <we...@gmail.com> 于2020年3月30日周一 下午2:38写道：
> Hi everyone,
>
> Are there more comments about this FLIP? If not, I would like to bring up
> the VOTE.
>
> Best,
> Wei
>
> > 在 2020年3月9日，23:18，Xingbo Huang <hx...@gmail.com> 写道：
> >
> > Hi Godfrey,
> > thanks for your suggestion.
> > I have added two examples how to use python UDF
> > in SQL and how to start sql-client.sh with full python dependencies In
> FLIP.
> >
> > Best,
> > Xingo
> >
> > godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
> >
> >> Hi Wei, thanks for the proposal.
> >>
> >> I think it's better to give two more examples, one is how to use python
> UDF
> >> in SQL, another is how to start sql-client.sh with full python
> >> dependencies.
> >>
> >> Best,
> >> Godfrey
> >>
> >> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
> >>
> >>> Hi everyone,
> >>>
> >>> I would like to start discussion about how to support Python UDF in SQL
> >>> Client.
> >>>
> >>> Flink Python UDF(FLIP-58[1]) has already been introduced in the release
> >> of
> >>> 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
> >>>
> >>> SQL Client defines UDF via the environment file and has its own CLI
> >>> implementation to manage dependencies, but neither of which supports
> >> Python
> >>> UDF. We want to introduce the support of Python UDF for SQL Client,
> >>> including the registration and the dependency management of Python UDF.
> >>>
> >>> Here is the design doc:
> >>>
> >>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
> >>>
> >>> Looking forward to your feedback!
> >>>
> >>> Best,
> >>> Wei
> >>>
> >>> [1]
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
> >>> [2]
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
> >>>
> >>>
> >>
>
>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by Wei Zhong <we...@gmail.com>.
Hi everyone,
Are there more comments about this FLIP? If not, I would like to bring up the VOTE.
Best,
Wei
> 在 2020年3月9日，23:18，Xingbo Huang <hx...@gmail.com> 写道：
>
> Hi Godfrey,
> thanks for your suggestion.
> I have added two examples how to use python UDF
> in SQL and how to start sql-client.sh with full python dependencies In FLIP.
>
> Best,
> Xingo
>
> godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
>
>> Hi Wei, thanks for the proposal.
>>
>> I think it's better to give two more examples, one is how to use python UDF
>> in SQL, another is how to start sql-client.sh with full python
>> dependencies.
>>
>> Best,
>> Godfrey
>>
>> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
>>
>>> Hi everyone,
>>>
>>> I would like to start discussion about how to support Python UDF in SQL
>>> Client.
>>>
>>> Flink Python UDF(FLIP-58[1]) has already been introduced in the release
>> of
>>> 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
>>>
>>> SQL Client defines UDF via the environment file and has its own CLI
>>> implementation to manage dependencies, but neither of which supports
>> Python
>>> UDF. We want to introduce the support of Python UDF for SQL Client,
>>> including the registration and the dependency management of Python UDF.
>>>
>>> Here is the design doc:
>>>
>>>
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
>>>
>>> Looking forward to your feedback!
>>>
>>> Best,
>>> Wei
>>>
>>> [1]
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
>>> [2]
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
>>>
>>>
>>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by Xingbo Huang <hx...@gmail.com>.
Hi Godfrey,
thanks for your suggestion.
I have added two examples how to use python UDF
in SQL and how to start sql-client.sh with full python dependencies In FLIP.
Best,
Xingo
godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
> Hi Wei, thanks for the proposal.
>
> I think it's better to give two more examples, one is how to use python UDF
> in SQL, another is how to start sql-client.sh with full python
> dependencies.
>
> Best,
> Godfrey
>
> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
>
> > Hi everyone,
> >
> > I would like to start discussion about how to support Python UDF in SQL
> > Client.
> >
> > Flink Python UDF(FLIP-58[1]) has already been introduced in the release
> of
> > 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
> >
> > SQL Client defines UDF via the environment file and has its own CLI
> > implementation to manage dependencies, but neither of which supports
> Python
> > UDF. We want to introduce the support of Python UDF for SQL Client,
> > including the registration and the dependency management of Python UDF.
> >
> > Here is the design doc:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
> >
> > Looking forward to your feedback!
> >
> > Best,
> > Wei
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
> > [2]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
> >
> >
>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by Xingbo Huang <hx...@gmail.com>.
Hi Wei,
Thanks a lot for the proposal.
Best,
Xingbo
godfrey he <go...@gmail.com> 于2020年3月9日周一 下午10:24写道：
> Hi Wei, thanks for the proposal.
>
> I think it's better to give two more examples, one is how to use python UDF
> in SQL, another is how to start sql-client.sh with full python
> dependencies.
>
> Best,
> Godfrey
>
> Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
>
> > Hi everyone,
> >
> > I would like to start discussion about how to support Python UDF in SQL
> > Client.
> >
> > Flink Python UDF(FLIP-58[1]) has already been introduced in the release
> of
> > 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
> >
> > SQL Client defines UDF via the environment file and has its own CLI
> > implementation to manage dependencies, but neither of which supports
> Python
> > UDF. We want to introduce the support of Python UDF for SQL Client,
> > including the registration and the dependency management of Python UDF.
> >
> > Here is the design doc:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
> >
> > Looking forward to your feedback!
> >
> > Best,
> > Wei
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
> > [2]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
> >
> >
>
Re: [DISCUSS] FLIP-114: Support Python UDF in SQL Client
Posted by godfrey he <go...@gmail.com>.
Hi Wei, thanks for the proposal.
I think it's better to give two more examples, one is how to use python UDF
in SQL, another is how to start sql-client.sh with full python dependencies.
Best,
Godfrey
Wei Zhong <we...@gmail.com> 于2020年3月9日周一 下午10:09写道：
> Hi everyone,
>
> I would like to start discussion about how to support Python UDF in SQL
> Client.
>
> Flink Python UDF(FLIP-58[1]) has already been introduced in the release of
> 1.10.0 and the support for SQL DDL is introduced in FLIP-106[2].
>
> SQL Client defines UDF via the environment file and has its own CLI
> implementation to manage dependencies, but neither of which supports Python
> UDF. We want to introduce the support of Python UDF for SQL Client,
> including the registration and the dependency management of Python UDF.
>
> Here is the design doc:
>
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-114%3A+Support+Python+UDF+in+SQL+Client
>
> Looking forward to your feedback!
>
> Best,
> Wei
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
> [2]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL
>
>