You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Harish Jaiprakash (JIRA)" <ji...@apache.org> on 2016/08/11 09:35:20 UTC
[jira] [Commented] (TEZ-3407) Tez with Alluxio not working
[ https://issues.apache.org/jira/browse/TEZ-3407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416947#comment-15416947 ]
Harish Jaiprakash commented on TEZ-3407:
----------------------------------------
Hi Ashish,
I'm not sure if you should be filing a jira for this. You should try the tez users mailing list first: https://tez.apache.org/mail-lists.html. And only file a bug if its really a bug.
You can see that there is java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64, which can occur if you have a version of apache-commons-codec less than 1.4. Allixio might have apache-commons-codec version 1.3, I'm not sure, I just did a cursory look on their pom.xml. You should try one of the following:
* export HADOOP_USER_CLASSPATH_FIRST=true; export HADOOP_CLASSPATH=<path-to-higher-version-apache-commons>
* Replace the apache-commons-codec in Allixio with your own and try it again.
* Ask Allixio to ship with a higher version of apache-commons-codec.
> Tez with Alluxio not working
> ----------------------------
>
> Key: TEZ-3407
> URL: https://issues.apache.org/jira/browse/TEZ-3407
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Ashish Kumar
>
> I started using Hive with Alluxio. And after few initial hurdles I'm able to make it work. Now, Hive with MR is working fine but the same query failing with Hive on TEZ engine.
> Hive Version 1.2
> Alluxio Version 1.0.1
> TEZ version 0.7
> Below is the error log:
> Error:
> org.apache.tez.dag.api.TezUncheckedException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:80)
> at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:98)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:137)
> at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:114)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4392)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.access$4300(VertexImpl.java:199)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:3233)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3183)
> at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3164)
> at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1812)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:198)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1943)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1929)
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
> at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:69)
> ... 20 more
> Caused by: java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.decodeBase64(Ljava/lang/String;)[B
> at org.apache.hadoop.hive.ql.exec.Utilities.getBaseWork(Utilities.java:419)
> at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:308)
> at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:92)
> ... 25 more
> 2016-08-11 08:37:57,399 ERROR [Dispatcher thread: Central] impl.VertexImpl: Can't handle Invalid event V_START on vertex Map 1 with vertexId vertex_1470806535809_0006_1_00 at current state NEW
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_START at NEW
> at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
> at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1812)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:198)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1943)
> at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1929)
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
> at java.lang.Thread.run(Thread.java:745)
> Anybody have any idea?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)