[HIVE-LLAP ]Tez Shuffle Handler

[Manoj Kumar <ma...@bigzetta.com>]

Hi all,

 I am trying to enable tez auxiliary service tez shuffler.
From hadoop logs it's confirmed that tez_shuffle is running

>  INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices:
> Adding auxiliary service tez_shuffle, "tez_shuffle"
> INFO org.apache.tez.auxservices.ShuffleHandler: tez_shuffle listening on
> port 13563
>

tez-site.xml
-------------
...
<property>
  <name>tez.am.shuffle.auxiliary-service.id</name>
  <value>tez_shuffle</value>
</property>

yarn-site.xml
-------------
...
<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>tez_shuffle</value>
</property>

<property>
  <name>yarn.nodemanager.aux-services.tez_shuffle.class</name>
  <value>org.apache.tez.auxservices.ShuffleHandler</value>
</property>


* On hive-llap I am getting below error:-*
TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) :
attempt_1594912627497_0009_1_01_000002_3:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:300)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:251)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:363)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
... 15 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async
Initialization failed. abortRequested=false
at
org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:466)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:400)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:573)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:525)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:386)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335)
... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:118)
at
org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache$1.call(LlapObjectCache.java:143)
... 4 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:280)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:327)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:396)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:203)
at
org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:116)
... 5 more


*Caused by: java.lang.NullPointerException at
org.apache.tez.runtime.api.impl.TezTaskContextImpl.getServiceConsumerMetaData(TezTaskContextImpl.java:190)
at
org.apache.tez.runtime.library.common.shuffle.impl.ShuffleManager.<init>(ShuffleManager.java:264)*
at
org.apache.tez.runtime.library.input.UnorderedKVInput.start(UnorderedKVInput.java:146)
at
org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:277)
... 9 more

Regards,
Manoj  Kumar