Hive Custom Simple Edge NullPointerException

[Bernard Quizon <be...@cheetahdigital.com>]

Hi.

I'm using Hive 3.1.0 (Tez Execution Engine) and I'm running into this NPE:

INFO  : Dag name: WITH event_agg AS (WITH outcome AS (SE...ASC (Stage-1)
ERROR : Failed to execute tez graph.
java.lang.NullPointerException: null
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.setupQuickStart(DagUtils.java:1551)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.exec.tez.DagUtils.createEdge(DagUtils.java:459)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:487)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:209)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:210)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2701)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2372)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2048)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1746)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1740)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
~[hive-exec-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:318)
~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_112]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_112]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
~[hadoop-common-3.1.1.3.0.1.0-187.jar:?]
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:331)
~[hive-service-3.1.0.3.0.1.0-187.jar:3.1.0.3.0.1.0-187]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_112]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_112]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[?:1.8.0_112]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
ERROR : FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.tez.TezTask


Somehow conversion from TezWork to Tez DAG throws a NullPointerException.
Has someone experienced a similar error?

BTW the query is pretty complicated, one with multiple left joins and CTEs.
It works on LLAP but fails on default Hive Server2.

Thanks!