You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/06/03 07:23:43 UTC
[GitHub] [incubator-dolphinscheduler] EthanLiu1987 opened a new issue #2888: [BUG] can not use hive udf
EthanLiu1987 opened a new issue #2888:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2888
*For better global communication, please give priority to using English description, thx! *
**Describe the bug**
1、upload a hive udf in resource center successfully
2、create a work flow , use this udf to query , but return error。
3、dolphin version is 1.2.0 release,hive version is 1.2.1 and 2.7.3 both test failed
4、we test the udf in the hive cli, query successfully,so the jar is good
**To Reproduce**
Steps to reproduce the behavior, for example:
1. upload a hive udf in resource center
2. create a work flow , use this udf to query
3. See error，error info :
[INFO] 2020-06-03 15:03:54.560 - [taskAppId=TASK-8-133-287]:[109] - SqlParameters{type='HIVE', datasource=2, sql='select name,id,idcardcheck(id) as result from ods_people', sqlType=0, udfs='', showType='TABLE', connParams='', title='test', receivers='fsdfs@asiainfo.com', receiversCc='', preStatements=[], postStatements=[]}
[INFO] 2020-06-03 15:03:54.567 - [taskAppId=TASK-8-133-287]:[110] - sql type : HIVE, datasource : 2, sql : select name,id,idcardcheck(id) as result from ods_people , localParams : [],udfs : ,showType : TABLE,connParams :
[INFO] 2020-06-03 15:03:54.570 - [taskAppId=TASK-8-133-287]:[127] - datasource name : hivetest2 , type : HIVE , desc : , user_id : 2 , parameter : {"address":"jdbc:hive2://10.19.83.39:10000","database":"default","jdbcUrl":"jdbc:hive2://10.19.83.39:10000/default","user":"hadoop","password":"hadoop"}
[INFO] 2020-06-03 15:03:54.571 - [taskAppId=TASK-8-133-287]:[214] - SQL tile : test
[INFO] 2020-06-03 15:03:54.572 - [taskAppId=TASK-8-133-287]:[448] - after replace sql , preparing : select name,id,idcardcheck(id) as result from ods_people
[INFO] 2020-06-03 15:03:54.572 - [taskAppId=TASK-8-133-287]:[453] - replaced sql , parameters:
[INFO] 2020-06-03 15:03:54.904 - [taskAppId=TASK-8-133-287]:[364] - prepare statement replace sql : org.apache.hive.jdbc.HivePreparedStatement@67d16b19
[ERROR] 2020-06-03 15:03:54.977 - [taskAppId=TASK-8-133-287]:[336] - Error while compiling statement: FAILED: SemanticException [Error 10011]: Line 1:15 Invalid function 'idcardcheck'
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException [Error 10011]: Line 1:15 Invalid function 'idcardcheck'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:264)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:250)
at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:309)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:250)
at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:434)
at org.apache.hive.jdbc.HivePreparedStatement.executeQuery(HivePreparedStatement.java:109)
at org.apache.dolphinscheduler.server.worker.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:295)
at org.apache.dolphinscheduler.server.worker.task.sql.SqlTask.handle(SqlTask.java:176)
at org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:142)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException [Error 10011]: Line 1:15 Invalid function 'idcardcheck'
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:112)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy20.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
... 3 common frames omitted
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.SemanticException:Line 1:15 Invalid function 'idcardcheck'
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:925)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1265)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:133)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:205)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:149)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:10512)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:10468)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3840)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3619)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8956)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8911)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9756)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9649)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10122)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:325)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10133)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
... 26 common frames omitted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong closed issue #2888: [BUG] can not use hive udf
Posted by GitBox <gi...@apache.org>.
dailidong closed issue #2888:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2888
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #2888: [BUG] can not use hive udf
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #2888:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2888#issuecomment-638551925
You can refer to this issue
https://github.com/apache/incubator-dolphinscheduler/issues/2435
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org