You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by Yaoting Gong <fa...@gmail.com> on 2020/04/02 03:22:04 UTC
Flink 1.10.0 HiveModule 函数问题
大家后，
我们项目目前 集成了HiveModule，遇到一些问题。请教下大家。
在集成 Hive Module 之前，substr，split 都是无法使用的。集成后，验证都是可以的。
比如：select split('1,2,2,4',',')
但是在使用 explode 函数，select explode(split('1,2,2,4',','));
有如下错误：
The main method caused an error: SQL validation failed. From line 1, column
8 to line 1, column 36: *No match found for function signature explode(*
*) *
*完整堆栈：*
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: SQL validation failed. From line 1, column 8 to line 1,
column 36: No match found for function signature explode()
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
at
org.apache.flink.client.cli.CliFrontend$$Lambda$38/1205406622.call(Unknown
Source)
at
org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$39/293907205.run(Unknown
Source)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
Caused by: org.apache.flink.table.api.ValidationException: SQL validation
failed. From line 1, column 8 to line 1, column 36: No match found for
function signature explode()
at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
$apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130)
at
org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105)
at
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127)
at
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlQuery(TableEnvironmentImpl.java:464)
at com.sui.bigdata.PlatformEngine$.$anonfun$main$4(PlatformEngine.scala:88)
at
com.sui.bigdata.PlatformEngine$.$anonfun$main$4$adapted(PlatformEngine.scala:87)
at com.sui.bigdata.PlatformEngine$$$Lambda$765/623545006.apply(Unknown
Source)
at scala.collection.immutable.List.foreach(List.scala:388)
at com.sui.bigdata.PlatformEngine$.main(PlatformEngine.scala:87)
at com.sui.bigdata.PlatformEngine.main(PlatformEngine.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
... 13 more
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1,
column 8 to line 1, column 36: No match found for function signature
explode()
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1773)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:293)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
at
org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:479)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4105)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3389)
at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1008)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:968)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943)
at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:650)
at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
$apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:126)
... 28 more
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match
found for function signature explode()
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
... 50 more
简单看了 Hive Functions 加载函数代码，
1、都进行了加载
system.registerGenericUDF("split", GenericUDFSplit.class);
system.registerGenericUDTF("explode", GenericUDTFExplode.class);
2、FunctionInfo 返回是否 builtIn 逻辑也是一致的
this.functionType = isNative ? FunctionType.BUILTIN :
FunctionType.TEMPORARY;， 其中 isNative 是单例对象的属性，
// registry for system functions
private static final Registry system = new Registry(true); // isNative
=== true
3、不在 Hive Module 黑名单中，附黑名单列表：
"dense_rank",
"first_value",
"lag",
"last_value",
"lead",
"rank",
"row_number",
"hop",
"hop_end",
"hop_proctime",
"hop_rowtime",
"hop_start",
"session",
"session_end",
"session_proctime",
"session_rowtime",
"session_start",
"tumble",
"tumble_end",
"tumble_proctime",
"tumble_rowtime",
"tumble_start"
不知道是不是还有什么地方使用方式不对？
谢谢解答。
Re: Flink 1.10.0 HiveModule 函数问题
Posted by Yaoting Gong <fa...@gmail.com>.
Hi, Jingsong Li
你好，非常感谢你的解答。问题已经得到处理。
类似写法之前有尝试过，但是没有注意到与hive的差异，包括 T() 这些，所以也是失败的。加上官方文档没有仔细理解。
再次感谢答疑
On Thu, Apr 2, 2020 at 11:27 AM Jingsong Li <ji...@gmail.com> wrote:
> Hi,
>
> GenericUDTFExplode是一个UDTF。
> Flink中使用UDTF的方式是标准SQL的方式：
> "select x from db1.nested, lateral table(explode(a)) as T(x)"
>
> 你试下。
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/udfs.html#table-functions
>
> Best,
> Jingsong Lee
>
> On Thu, Apr 2, 2020 at 11:22 AM Yaoting Gong <fa...@gmail.com>
> wrote:
>
> > 大家后，
> >
> > 我们项目目前 集成了HiveModule，遇到一些问题。请教下大家。
> >
> > 在集成 Hive Module 之前，substr，split 都是无法使用的。集成后，验证都是可以的。
> > 比如：select split('1,2,2,4',',')
> >
> > 但是在使用 explode 函数，select explode(split('1,2,2,4',','));
> > 有如下错误：
> >
> >
> > The main method caused an error: SQL validation failed. From line 1,
> column
> > 8 to line 1, column 36: *No match found for function signature explode(*
> > *) *
> >
> > *完整堆栈：*
> > The program finished with the following exception:
> >
> > org.apache.flink.client.program.ProgramInvocationException: The main
> method
> > caused an error: SQL validation failed. From line 1, column 8 to line 1,
> > column 36: No match found for function signature explode()
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
> > at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
> > at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
> > at
> >
> >
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
> > at
> >
> org.apache.flink.client.cli.CliFrontend$$Lambda$38/1205406622.call(Unknown
> > Source)
> > at
> >
> >
> org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$39/293907205.run(Unknown
> > Source)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> > at
> >
> >
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
> > Caused by: org.apache.flink.table.api.ValidationException: SQL validation
> > failed. From line 1, column 8 to line 1, column 36: No match found for
> > function signature explode()
> > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
> >
> >
> $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130)
> > at
> >
> >
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105)
> > at
> >
> >
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127)
> > at
> >
> >
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
> > at
> >
> >
> org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlQuery(TableEnvironmentImpl.java:464)
> > at
> com.sui.bigdata.PlatformEngine$.$anonfun$main$4(PlatformEngine.scala:88)
> > at
> >
> >
> com.sui.bigdata.PlatformEngine$.$anonfun$main$4$adapted(PlatformEngine.scala:87)
> > at com.sui.bigdata.PlatformEngine$$$Lambda$765/623545006.apply(Unknown
> > Source)
> > at scala.collection.immutable.List.foreach(List.scala:388)
> > at com.sui.bigdata.PlatformEngine$.main(PlatformEngine.scala:87)
> > at com.sui.bigdata.PlatformEngine.main(PlatformEngine.scala)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:497)
> > at
> >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
> > ... 13 more
> > Caused by: org.apache.calcite.runtime.CalciteContextException: From line
> 1,
> > column 8 to line 1, column 36: No match found for function signature
> > explode()
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at
> >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> > at
> >
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
> > at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1773)
> > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:293)
> > at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
> > at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:479)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4105)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3389)
> > at
> >
> >
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> > at
> >
> >
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1008)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:968)
> > at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943)
> > at
> >
> >
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:650)
> > at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
> >
> >
> $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:126)
> > ... 28 more
> > Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No
> match
> > found for function signature explode()
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at
> >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at
> >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> > at
> >
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> > at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
> > ... 50 more
> >
> >
> > 简单看了 Hive Functions 加载函数代码，
> > 1、都进行了加载
> > system.registerGenericUDF("split", GenericUDFSplit.class);
> > system.registerGenericUDTF("explode", GenericUDTFExplode.class);
> >
> > 2、FunctionInfo 返回是否 builtIn 逻辑也是一致的
> > this.functionType = isNative ? FunctionType.BUILTIN :
> > FunctionType.TEMPORARY;， 其中 isNative 是单例对象的属性，
> > // registry for system functions
> > private static final Registry system = new Registry(true); //
> isNative
> > === true
> >
> > 3、不在 Hive Module 黑名单中，附黑名单列表：
> > "dense_rank",
> > "first_value",
> > "lag",
> > "last_value",
> > "lead",
> > "rank",
> > "row_number",
> > "hop",
> > "hop_end",
> > "hop_proctime",
> > "hop_rowtime",
> > "hop_start",
> > "session",
> > "session_end",
> > "session_proctime",
> > "session_rowtime",
> > "session_start",
> > "tumble",
> > "tumble_end",
> > "tumble_proctime",
> > "tumble_rowtime",
> > "tumble_start"
> >
> > 不知道是不是还有什么地方使用方式不对？
> >
> > 谢谢解答。
> >
>
>
> --
> Best, Jingsong Lee
>
Re: Flink 1.10.0 HiveModule 函数问题
Posted by Jingsong Li <ji...@gmail.com>.
Hi,
GenericUDTFExplode是一个UDTF。
Flink中使用UDTF的方式是标准SQL的方式：
"select x from db1.nested, lateral table(explode(a)) as T(x)"
你试下。
[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/udfs.html#table-functions
Best,
Jingsong Lee
On Thu, Apr 2, 2020 at 11:22 AM Yaoting Gong <fa...@gmail.com>
wrote:
> 大家后，
>
> 我们项目目前 集成了HiveModule，遇到一些问题。请教下大家。
>
> 在集成 Hive Module 之前，substr，split 都是无法使用的。集成后，验证都是可以的。
> 比如：select split('1,2,2,4',',')
>
> 但是在使用 explode 函数，select explode(split('1,2,2,4',','));
> 有如下错误：
>
>
> The main method caused an error: SQL validation failed. From line 1, column
> 8 to line 1, column 36: *No match found for function signature explode(*
> *) *
>
> *完整堆栈：*
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: SQL validation failed. From line 1, column 8 to line 1,
> column 36: No match found for function signature explode()
> at
>
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
> at
>
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
> at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
> at
>
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
> at
>
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)
> at
>
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)
> at
> org.apache.flink.client.cli.CliFrontend$$Lambda$38/1205406622.call(Unknown
> Source)
> at
>
> org.apache.flink.runtime.security.HadoopSecurityContext$$Lambda$39/293907205.run(Unknown
> Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> at
>
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)
> Caused by: org.apache.flink.table.api.ValidationException: SQL validation
> failed. From line 1, column 8 to line 1, column 36: No match found for
> function signature explode()
> at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
>
> $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:130)
> at
>
> org.apache.flink.table.planner.calcite.FlinkPlannerImpl.validate(FlinkPlannerImpl.scala:105)
> at
>
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:127)
> at
>
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:66)
> at
>
> org.apache.flink.table.api.internal.TableEnvironmentImpl.sqlQuery(TableEnvironmentImpl.java:464)
> at com.sui.bigdata.PlatformEngine$.$anonfun$main$4(PlatformEngine.scala:88)
> at
>
> com.sui.bigdata.PlatformEngine$.$anonfun$main$4$adapted(PlatformEngine.scala:87)
> at com.sui.bigdata.PlatformEngine$$$Lambda$765/623545006.apply(Unknown
> Source)
> at scala.collection.immutable.List.foreach(List.scala:388)
> at com.sui.bigdata.PlatformEngine$.main(PlatformEngine.scala:87)
> at com.sui.bigdata.PlatformEngine.main(PlatformEngine.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
> ... 13 more
> Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1,
> column 8 to line 1, column 36: No match found for function signature
> explode()
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
> at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4841)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1773)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:293)
> at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:219)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5600)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5587)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1691)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1676)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:479)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4105)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3389)
> at
>
> org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
> at
>
> org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1008)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:968)
> at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:943)
> at
>
> org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:650)
> at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org
>
> $apache$flink$table$planner$calcite$FlinkPlannerImpl$$validate(FlinkPlannerImpl.scala:126)
> ... 28 more
> Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match
> found for function signature explode()
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
>
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
> ... 50 more
>
>
> 简单看了 Hive Functions 加载函数代码，
> 1、都进行了加载
> system.registerGenericUDF("split", GenericUDFSplit.class);
> system.registerGenericUDTF("explode", GenericUDTFExplode.class);
>
> 2、FunctionInfo 返回是否 builtIn 逻辑也是一致的
> this.functionType = isNative ? FunctionType.BUILTIN :
> FunctionType.TEMPORARY;， 其中 isNative 是单例对象的属性，
> // registry for system functions
> private static final Registry system = new Registry(true); // isNative
> === true
>
> 3、不在 Hive Module 黑名单中，附黑名单列表：
> "dense_rank",
> "first_value",
> "lag",
> "last_value",
> "lead",
> "rank",
> "row_number",
> "hop",
> "hop_end",
> "hop_proctime",
> "hop_rowtime",
> "hop_start",
> "session",
> "session_end",
> "session_proctime",
> "session_rowtime",
> "session_start",
> "tumble",
> "tumble_end",
> "tumble_proctime",
> "tumble_rowtime",
> "tumble_start"
>
> 不知道是不是还有什么地方使用方式不对？
>
> 谢谢解答。
>
--
Best, Jingsong Lee