You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Alan Sheinberg <as...@confluent.io.INVALID> on 2024/04/29 22:57:18 UTC
[Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of
Function Calls While Constant-folding [1]
This feature proposes adding a new
method FunctionDefinition.allowConstantFolding() as part of the Flink
Table/SQL API. This would be used to determine whether an expression
containing this function should have constant-folding logic run on it,
invoking the function at planning time.
The current behavior of always doing constant-folding on function calls is
problematic for UDFs which invoke RPCs or have other side effects in
external systems. In these cases, you either don’t want these actions to
occur during planning time, or it may be important to happen on a per
result row basis.
Note that this is a bit different than
FunctionDefinition.isDeterministic(), and can exist along-side it.
Looking forward to your feedback and suggestions.
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
Thanks,
Alan
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Alan Sheinberg <as...@confluent.io.INVALID>.
Thanks for the comment Timo!
Since FunctionDefinition is kind of a declaration, it should rather read
> declarative as well. I suggest to call this method
> `supportsConstantFolding`. `allow` sounds like one could influence the
> state of the function from the outside.
>
That makes sense. Changed to supportsConstantFolding to make this clearer.
-Alan
On Tue, Apr 30, 2024 at 5:35 AM Timo Walther <tw...@apache.org> wrote:
> Hi Alan,
>
> thanks for starting this small FLIP. A new method in FunctionDefinition
> makes a lot of sense and the lack of it was an issue in past already. We
> should clearly separate determinism and constant folding behavior.
>
> Since FunctionDefinition is kind of a declaration, it should rather read
> declarative as well. I suggest to call this method
> `supportsConstantFolding`. `allow` sounds like one could influence the
> state of the function from the outside.
>
> Otherwise +1 for the FLIP.
>
> Regards,
> Timo
>
>
> On 30.04.24 00:57, Alan Sheinberg wrote:
> > I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of
> > Function Calls While Constant-folding [1]
> >
> > This feature proposes adding a new
> > method FunctionDefinition.allowConstantFolding() as part of the Flink
> > Table/SQL API. This would be used to determine whether an expression
> > containing this function should have constant-folding logic run on it,
> > invoking the function at planning time.
> >
> > The current behavior of always doing constant-folding on function calls
> is
> > problematic for UDFs which invoke RPCs or have other side effects in
> > external systems. In these cases, you either don’t want these actions to
> > occur during planning time, or it may be important to happen on a per
> > result row basis.
> >
> > Note that this is a bit different than
> > FunctionDefinition.isDeterministic(), and can exist along-side it.
> >
> > Looking forward to your feedback and suggestions.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
> >
> >
> > Thanks,
> > Alan
> >
>
>
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Timo Walther <tw...@apache.org>.
Hi Alan,
thanks for starting this small FLIP. A new method in FunctionDefinition
makes a lot of sense and the lack of it was an issue in past already. We
should clearly separate determinism and constant folding behavior.
Since FunctionDefinition is kind of a declaration, it should rather read
declarative as well. I suggest to call this method
`supportsConstantFolding`. `allow` sounds like one could influence the
state of the function from the outside.
Otherwise +1 for the FLIP.
Regards,
Timo
On 30.04.24 00:57, Alan Sheinberg wrote:
> I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of
> Function Calls While Constant-folding [1]
>
> This feature proposes adding a new
> method FunctionDefinition.allowConstantFolding() as part of the Flink
> Table/SQL API. This would be used to determine whether an expression
> containing this function should have constant-folding logic run on it,
> invoking the function at planning time.
>
> The current behavior of always doing constant-folding on function calls is
> problematic for UDFs which invoke RPCs or have other side effects in
> external systems. In these cases, you either don’t want these actions to
> occur during planning time, or it may be important to happen on a per
> result row basis.
>
> Note that this is a bit different than
> FunctionDefinition.isDeterministic(), and can exist along-side it.
>
> Looking forward to your feedback and suggestions.
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
>
>
> Thanks,
> Alan
>
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Lincoln Lee <li...@gmail.com>.
Thanks Alan for starting this flip!
+1 for the new `supportsConstantFolding`. We do need to reserve the
ability to not perform constant folding for certain cases.
Best,
Lincoln Lee
Alan Sheinberg <as...@confluent.io.invalid> 于2024年5月7日周二 08:32写道：
> Hi Muhammet, Timo
>
> The `isDeterministic()`[1] function is mentioned in the documentation,
> > I would suggest to add maybe a section for `supportsConstantFolding()`,
> > with short description and examples use cases (similar to the
> > motivation of the FLIP) where this could be useful in UDFs.
> >
> Thanks for the suggestion. I'll do that.
>
> Feel free to start a vote thread by tomorrow. If there are no objections?
>
> Sounds good. I'll do that tomorrow if I hear no objections by then.
>
> Thanks,
> Alan
>
> On Mon, May 6, 2024 at 8:12 AM Timo Walther <tw...@apache.org> wrote:
>
> > Hi Alan,
> >
> > thanks for the update. From my side, the FLIP seems good for voting.
> >
> > Since it only touches a small API surface, I guess the proposal is not
> > very controversial.
> >
> > Feel free to start a vote thread by tomorrow. If there are no objections?
> >
> > Thanks,
> > Timo
> >
> >
> > On 02.05.24 09:45, Muhammet Orazov wrote:
> > > Hey Alan,
> > >
> > > Thanks for the proposal, +1!
> > >
> > > The `isDeterministic()`[1] function is mentioned in the documentation,
> > > I would suggest to add maybe a section for `supportsConstantFolding()`,
> > > with short description and examples use cases (similar to the
> > > motivation of the FLIP) where this could be useful in UDFs.
> > >
> > > Best,
> > > Muhammet
> > >
> > > [1]:
> > >
> >
> https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/table/functions/udfs/#evaluation-methods
> > >
> > >
> > > On 2024-04-29 22:57, Alan Sheinberg wrote:
> > >> I'd like to start a discussion of FLIP-452: Allow Skipping Invocation
> of
> > >> Function Calls While Constant-folding [1]
> > >>
> > >> This feature proposes adding a new
> > >> method FunctionDefinition.allowConstantFolding() as part of the Flink
> > >> Table/SQL API. This would be used to determine whether an expression
> > >> containing this function should have constant-folding logic run on it,
> > >> invoking the function at planning time.
> > >>
> > >> The current behavior of always doing constant-folding on function
> > >> calls is
> > >> problematic for UDFs which invoke RPCs or have other side effects in
> > >> external systems. In these cases, you either don’t want these actions
> > to
> > >> occur during planning time, or it may be important to happen on a per
> > >> result row basis.
> > >>
> > >> Note that this is a bit different than
> > >> FunctionDefinition.isDeterministic(), and can exist along-side it.
> > >>
> > >> Looking forward to your feedback and suggestions.
> > >>
> > >> [1]
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
> > >>
> > >>
> > >> Thanks,
> > >> Alan
> > >
> >
> >
>
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Alan Sheinberg <as...@confluent.io.INVALID>.
Hi Muhammet, Timo
The `isDeterministic()`[1] function is mentioned in the documentation,
> I would suggest to add maybe a section for `supportsConstantFolding()`,
> with short description and examples use cases (similar to the
> motivation of the FLIP) where this could be useful in UDFs.
>
Thanks for the suggestion. I'll do that.
Feel free to start a vote thread by tomorrow. If there are no objections?
Sounds good. I'll do that tomorrow if I hear no objections by then.
Thanks,
Alan
On Mon, May 6, 2024 at 8:12 AM Timo Walther <tw...@apache.org> wrote:
> Hi Alan,
>
> thanks for the update. From my side, the FLIP seems good for voting.
>
> Since it only touches a small API surface, I guess the proposal is not
> very controversial.
>
> Feel free to start a vote thread by tomorrow. If there are no objections?
>
> Thanks,
> Timo
>
>
> On 02.05.24 09:45, Muhammet Orazov wrote:
> > Hey Alan,
> >
> > Thanks for the proposal, +1!
> >
> > The `isDeterministic()`[1] function is mentioned in the documentation,
> > I would suggest to add maybe a section for `supportsConstantFolding()`,
> > with short description and examples use cases (similar to the
> > motivation of the FLIP) where this could be useful in UDFs.
> >
> > Best,
> > Muhammet
> >
> > [1]:
> >
> https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/table/functions/udfs/#evaluation-methods
> >
> >
> > On 2024-04-29 22:57, Alan Sheinberg wrote:
> >> I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of
> >> Function Calls While Constant-folding [1]
> >>
> >> This feature proposes adding a new
> >> method FunctionDefinition.allowConstantFolding() as part of the Flink
> >> Table/SQL API. This would be used to determine whether an expression
> >> containing this function should have constant-folding logic run on it,
> >> invoking the function at planning time.
> >>
> >> The current behavior of always doing constant-folding on function
> >> calls is
> >> problematic for UDFs which invoke RPCs or have other side effects in
> >> external systems. In these cases, you either don’t want these actions
> to
> >> occur during planning time, or it may be important to happen on a per
> >> result row basis.
> >>
> >> Note that this is a bit different than
> >> FunctionDefinition.isDeterministic(), and can exist along-side it.
> >>
> >> Looking forward to your feedback and suggestions.
> >>
> >> [1]
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
> >>
> >>
> >> Thanks,
> >> Alan
> >
>
>
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Timo Walther <tw...@apache.org>.
Hi Alan,
thanks for the update. From my side, the FLIP seems good for voting.
Since it only touches a small API surface, I guess the proposal is not
very controversial.
Feel free to start a vote thread by tomorrow. If there are no objections?
Thanks,
Timo
On 02.05.24 09:45, Muhammet Orazov wrote:
> Hey Alan,
>
> Thanks for the proposal, +1!
>
> The `isDeterministic()`[1] function is mentioned in the documentation,
> I would suggest to add maybe a section for `supportsConstantFolding()`,
> with short description and examples use cases (similar to the
> motivation of the FLIP) where this could be useful in UDFs.
>
> Best,
> Muhammet
>
> [1]:
> https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/table/functions/udfs/#evaluation-methods
>
>
> On 2024-04-29 22:57, Alan Sheinberg wrote:
>> I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of
>> Function Calls While Constant-folding [1]
>>
>> This feature proposes adding a new
>> method FunctionDefinition.allowConstantFolding() as part of the Flink
>> Table/SQL API. This would be used to determine whether an expression
>> containing this function should have constant-folding logic run on it,
>> invoking the function at planning time.
>>
>> The current behavior of always doing constant-folding on function
>> calls is
>> problematic for UDFs which invoke RPCs or have other side effects in
>> external systems. In these cases, you either don’t want these actions to
>> occur during planning time, or it may be important to happen on a per
>> result row basis.
>>
>> Note that this is a bit different than
>> FunctionDefinition.isDeterministic(), and can exist along-side it.
>>
>> Looking forward to your feedback and suggestions.
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
>>
>>
>> Thanks,
>> Alan
>
Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding
Posted by Muhammet Orazov <mo...@morazow.com.INVALID>.
Hey Alan,
Thanks for the proposal, +1!
The `isDeterministic()`[1] function is mentioned in the documentation,
I would suggest to add maybe a section for `supportsConstantFolding()`,
with short description and examples use cases (similar to the
motivation of the FLIP) where this could be useful in UDFs.
Best,
Muhammet
[1]:
https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/table/functions/udfs/#evaluation-methods
On 2024-04-29 22:57, Alan Sheinberg wrote:
> I'd like to start a discussion of FLIP-452: Allow Skipping Invocation
> of
> Function Calls While Constant-folding [1]
>
> This feature proposes adding a new
> method FunctionDefinition.allowConstantFolding() as part of the Flink
> Table/SQL API. This would be used to determine whether an expression
> containing this function should have constant-folding logic run on it,
> invoking the function at planning time.
>
> The current behavior of always doing constant-folding on function calls
> is
> problematic for UDFs which invoke RPCs or have other side effects in
> external systems. In these cases, you either don’t want these actions
> to
> occur during planning time, or it may be important to happen on a per
> result row basis.
>
> Note that this is a bit different than
> FunctionDefinition.isDeterministic(), and can exist along-side it.
>
> Looking forward to your feedback and suggestions.
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-452%3A+Allow+Skipping+Invocation+of+Function+Calls+While+Constant-folding
>
>
> Thanks,
> Alan