You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by shamit jain <ja...@gmail.com> on 2021/07/13 18:56:50 UTC
Flink UDF Scalar Function called only once for all rows of select SQL in case of no parameter passed
Hi,
I am facing an issue where scalar UDF called once in case if no parameter passed as given below:-
public class DateTimeToEpochUDF extends ScalarFunction {
public Long eval() {
System.out.print("Test");
return Instant.now().toEpochMilli();
}
}
Now if I run the below query I will get the same time, looks like the function is called only one time, as the SOP also not printed.
select DateTimeToEpochUDF(), employee_ID from Employee;
Please note if I passed the parameter in UDF like primary key it is called separately for each record of employee.
Request you to please let me know if I am doing something wrong.
regards,
Shamit Jain
Re: Flink UDF Scalar Function called only once for all rows of select SQL in case of no parameter passed
Posted by shamit jain <ja...@gmail.com>.
Thanks!!
On 2021/07/14 02:26:47, JING ZHANG <be...@gmail.com> wrote:
> Hi, Shamit Jain,
> In fact, it is an optimization to simplify expression.
> If a Udf has no parameters, optimizer would be look it as an expression
> which always generate constants results.
> So it would be calculated once in optimization phase instead of run by per
> record in. runtime phase.
>
> The optimization would be disabled if any of the following condition is
> satisfied:
> (1) Expression has at least one parameter
> (2) Udf is not deterministic.(default value of `isDeterministic` is true,
> you could change the behavior by override the metjod `isDeterministic` of
> FunctionDefinition)
> (3) Udf is a dynamic function. (not applied for UDF, it is used for flink
> built-in SqlOperator).
>
> Best,
> JING ZHANG
>
>
> shamit jain <ja...@gmail.com> 于2021年7月14日周三 上午3:00写道：
>
> > Hi,
> >
> > I am facing an issue where scalar UDF called once in case if no parameter
> > passed as given below:-
> >
> > public class DateTimeToEpochUDF extends ScalarFunction {
> > public Long eval() {
> > System.out.print("Test");
> > return Instant.now().toEpochMilli();
> > }
> > }
> >
> > Now if I run the below query I will get the same time, looks like the
> > function is called only one time, as the SOP also not printed.
> >
> > select DateTimeToEpochUDF(), employee_ID from Employee;
> >
> > Please note if I passed the parameter in UDF like primary key it is called
> > separately for each record of employee.
> >
> > Request you to please let me know if I am doing something wrong.
> >
> > regards,
> > Shamit Jain
> >
>
Re: Flink UDF Scalar Function called only once for all rows of select
SQL in case of no parameter passed
Posted by JING ZHANG <be...@gmail.com>.
Hi, Shamit Jain,
In fact, it is an optimization to simplify expression.
If a Udf has no parameters, optimizer would be look it as an expression
which always generate constants results.
So it would be calculated once in optimization phase instead of run by per
record in. runtime phase.
The optimization would be disabled if any of the following condition is
satisfied:
(1) Expression has at least one parameter
(2) Udf is not deterministic.(default value of `isDeterministic` is true,
you could change the behavior by override the metjod `isDeterministic` of
FunctionDefinition)
(3) Udf is a dynamic function. (not applied for UDF, it is used for flink
built-in SqlOperator).
Best,
JING ZHANG
shamit jain <ja...@gmail.com> 于2021年7月14日周三 上午3:00写道：
> Hi,
>
> I am facing an issue where scalar UDF called once in case if no parameter
> passed as given below:-
>
> public class DateTimeToEpochUDF extends ScalarFunction {
> public Long eval() {
> System.out.print("Test");
> return Instant.now().toEpochMilli();
> }
> }
>
> Now if I run the below query I will get the same time, looks like the
> function is called only one time, as the SOP also not printed.
>
> select DateTimeToEpochUDF(), employee_ID from Employee;
>
> Please note if I passed the parameter in UDF like primary key it is called
> separately for each record of employee.
>
> Request you to please let me know if I am doing something wrong.
>
> regards,
> Shamit Jain
>
Re: Flink UDF Scalar Function called only once for all rows of select
SQL in case of no parameter passed
Posted by Caizhi Weng <ts...@gmail.com>.
Hi!
You should override the isDeterministic() method and return false. The
default return value of this method is true.
From the java doc of this method:
> Furthermore, return false if the planner should always execute this
> function on the cluster side. In other words: the planner should not
> perform constant expression reduction during planning for constant calls to
> this function.
shamit jain <ja...@gmail.com> 于2021年7月14日周三 上午2:56写道：
> Hi,
>
> I am facing an issue where scalar UDF called once in case if no parameter
> passed as given below:-
>
> public class DateTimeToEpochUDF extends ScalarFunction {
> public Long eval() {
> System.out.print("Test");
> return Instant.now().toEpochMilli();
> }
> }
>
> Now if I run the below query I will get the same time, looks like the
> function is called only one time, as the SOP also not printed.
>
> select DateTimeToEpochUDF(), employee_ID from Employee;
>
> Please note if I passed the parameter in UDF like primary key it is called
> separately for each record of employee.
>
> Request you to please let me know if I am doing something wrong.
>
> regards,
> Shamit Jain
>