You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Varun Rao <va...@cloudera.com> on 2019/06/18 01:37:49 UTC
Failing because I am unlikely to write too when querying UDF
Hello,
Previously we were able to run a query which queried a UDF without any issues.
We have made changes to the logic of the UDF so a jar was built, the
UDF function was dropped and the UDF function was recreated. However,
calling the same query which calls the UDF causes an issue.
Following are the steps too to readd the udf
DROP FUNCTION <name of udf>
CREATE FUNCTION <name of UDF> AS 'udf.scala_class_name' USING JAR
'hdfs:///tmp/udf_jar.jar'
INFO : Executing
command(queryId=hive_20190617162424_71f58164-dcb1-4003-8420-67773a7ea70c):
SELECT * FROM table_name
Below is the log file with the error, and it looks as if the udf jar
is causing the issue
WHERE
filter1 = filter1_value
AND filter2 = filter2_value
AND category LIKE 'filter3_value'
LIMIT 5
INFO : Query ID = hive_20190617162424_71f58164-dcb1-4003-8420-67773a7ea70c
INFO : Total jobs = 1
INFO : Launching Job 1 out of 1
INFO : Starting task [Stage-1:MAPRED] in serial mode
INFO : Session is already open
ERROR : Failed to execute tez graph.*java.io.IOException: Previous
writer likely failed to write
hdfs://host_name/tmp/hive/hive/_tez_session_dir/1ecc77ce-2db8-4127-aba5-1cfe9566d812/udf_jar.jar.
Failing because I am unlikely to write too.*
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:1093)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:972)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:899)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:488)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:358)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:175)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1414)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1211)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1204)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
[hive-service-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
[hive-service-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336)
[hive-service-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at java.security.AccessController.doPrivileged(Native Method)
[?:1.8.0_162]
at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_162]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
[hadoop-common-2.7.3.2.6.5.0-292.jar:?]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:350)
[hive-service-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_162]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_162]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_162]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_162]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_162]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_162]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
ERROR : FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.tez.TezTask
INFO : Resetting the caller context to
HIVE_SSN_ID:3a05b152-286e-4468-ab81-1143d233891a
INFO : Completed executing
command(queryId=hive_20190617162424_71f58164-dcb1-4003-8420-67773a7ea70c);
Time taken: 25.051 seconds
Error: Error while processing statement: FAILED: Execution Error,
return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
(state=08S01,code=1)
Thanks
Yours Truly,
Varun Rao