You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2020/10/13 09:10:00 UTC
[jira] [Resolved] (HIVE-24063) SqlFunctionConverter#getHiveUDF
handles cast before geting FunctionInfo
[ https://issues.apache.org/jira/browse/HIVE-24063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihua Deng resolved HIVE-24063.
--------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
> SqlFunctionConverter#getHiveUDF handles cast before geting FunctionInfo
> -----------------------------------------------------------------------
>
> Key: HIVE-24063
> URL: https://issues.apache.org/jira/browse/HIVE-24063
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Trivial
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When the current SqlOperator is SqlCastFunction, FunctionRegistry.getFunctionInfo would return null,
> but when hive.allow.udf.load.on.demand is enabled, HiveServer2 will refer to metastore for the function definition, an exception stack trace can be seen here in HiveServer2 log:
> INFO exec.FunctionRegistry: Unable to look up default.cast in metastore
> org.apache.hadoop.hive.ql.metadata.HiveException: NoSuchObjectException(message:Function @hive#default.cast does not exist)
> at org.apache.hadoop.hive.ql.metadata.Hive.getFunction(Hive.java:5495) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Registry.getFunctionInfoFromMetastoreNoLock(Registry.java:788) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Registry.getQualifiedFunctionInfo(Registry.java:657) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Registry.getFunctionInfo(Registry.java:351) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.FunctionRegistry.getFunctionInfo(FunctionRegistry.java:597) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.translator.SqlFunctionConverter.getHiveUDF(SqlFunctionConverter.java:158) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:112) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:68) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:134) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:68) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:134) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:68) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.PartitionPrune$ExtractPartPruningPredicate.visitCall(PartitionPrune.java:134) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
>
> So it's may be better to handle explicit cast before geting the FunctionInfo from Registry. Even if there is no cast in the query, the method handleExplicitCast returns null quickly when op.kind is not a SqlKind.CAST.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)