You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/12/23 01:14:00 UTC
[jira] [Resolved] (IMPALA-7877) Support Hive GenericUDF
[ https://issues.apache.org/jira/browse/IMPALA-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-7877.
-----------------------------------
Resolution: Duplicate
> Support Hive GenericUDF
> -----------------------
>
> Key: IMPALA-7877
> URL: https://issues.apache.org/jira/browse/IMPALA-7877
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.0
> Reporter: eugen yushin
> Priority: Major
>
> Running Hive UDF extending GenericUDF interface results in class cast exception. Relevant [code block|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java#L586]:
> {code}
> LOG.debug("Loading UDF '" + udfPath + "' from " + jarPath);
> loader = getClassLoader(jarPath);
> Class<?> c = Class.forName(udfPath, true, loader);
> Class<? extends UDF> udfClass = c.asSubclass(UDF.class);
> {code}
> Reproduce steps:
> {code}
> create function my_lower(string) returns string location '/path/to/hive-exec-1.1.0-cdh5.15.0.jar' symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower';
> select my_lower('Some String NOT ALREADY LOWERCASE');
> {code}
> Stack trace:
> {code}
> I1121 11:58:29.509138 29092 Frontend.java:952] Analyzing query: select my_lower('Some String NOT ALREADY LOWERCASE')
> I1121 11:58:29.513121 29092 UdfExecutor.java:581] Loading UDF 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower' from /var/lib/impala/udfs/hive-exec-1.1.0-cdh5.15.0.83728.2.jar
> I1121 11:58:29.515535 29092 jni-util.cc:230] java.lang.ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower
> at java.lang.Class.asSubclass(Class.java:3404)
> at org.apache.impala.hive.executor.UdfExecutor.init(UdfExecutor.java:584)
> at org.apache.impala.hive.executor.UdfExecutor.<init>(UdfExecutor.java:217)
> at org.apache.impala.service.FeSupport.NativeEvalExprsWithoutRow(Native Method)
> at org.apache.impala.service.FeSupport.EvalExprsWithoutRow(FeSupport.java:208)
> at org.apache.impala.service.FeSupport.EvalExprWithoutRow(FeSupport.java:163)
> at org.apache.impala.analysis.LiteralExpr.create(LiteralExpr.java:184)
> at org.apache.impala.rewrite.FoldConstantsRule.apply(FoldConstantsRule.java:68)
> at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
> at org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
> at org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
> at org.apache.impala.analysis.SelectList.rewriteExprs(SelectList.java:97)
> at org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:894)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:432)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:393)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:962)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156)
> I1121 11:58:29.523166 29092 status.cc:125] ClassCastException: class org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower
> @ 0x96663a impala::Status::Status()
> @ 0xcedfdd impala::JniUtil::GetJniExceptionMsg()
> @ 0x109457f impala::HiveUdfCall::OpenEvaluator()
> @ 0x96d757 impala::ScalarExprEvaluator::Open()
> @ 0xbedc2d Java_org_apache_impala_service_FeSupport_NativeEvalExprsWithoutRow
> @ 0x7fc705b49e6d (unknown)
> {code}
> Marked as bug because there're no any notes related to this behaviour in docs (while it claims Impala supports Hive UDF, it should support all possible Hive UDF formats if other is not specified).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)