You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Dharmesh Kakadia <dh...@gmail.com> on 2016/02/22 13:14:15 UTC
Re: Hive 2.0 + tez setting up
It looks like those classes are added after 0.7 version of Tez. I got past
that by using Tez version 0.8. May be we should add this to
https://cwiki.apache.org/confluence/display/Hive/Hive-Tez+Compatibility to
help others. or someone can give me permission to edit the wiki - my id is
dharmesh.kakadia:)
But now, I am stuck into "Incompatible versions found, clientVersion=0.8.2,
AMVersion=0.7.0" error. I tried also
setting tez.am.disable.client-version-check=true but still the same
problem. Any pointer would help. Also, where does tez AM Jar is picked from
?
Thanks,
Dharmesh
On Monday, February 22, 2016, Dharmesh Kakadia <dh...@gmail.com> wrote:
> Hi,
>
> The hive 2.0 is exciting. Thanks for the great work !
> I am getting the following error while giving it a try with tez 0.7.
>
> java.lang.NoClassDefFoundError:
> org/apache/tez/serviceplugins/api/TaskScheduler
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.<clinit>(TezSessionState.java:93)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.createAndInitSession(TezSessionPoolManager.java:209)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getNewSessionState(TezSessionPoolManager.java:258)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:238)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:415)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:343)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:134)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:222)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.tez.serviceplugins.api.TaskScheduler
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> ... 38 more
> FAILED: Execution Error, return code -101 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask.
> org/apache/tez/serviceplugins/api/TaskScheduler
>
> I understand that its not able to find a Tez class from the classpath, but
> could not understand why. I am using the same tez binaries with hive 1.2
> without any troubles. Is there are a required minimum version of Tez that
> hive 2.0 supports? I downloaded 2.0 release from the website and using it
> with 0.7 tez. Cluster is running latest hadoop/yarn.
>
> Thanks,
> Dharmesh
>
>
Re: Hive 2.0 + tez setting up
Posted by Siddharth Seth <ss...@apache.org>.
As part of using Tez-0.8, you will need to update the tez tar which resides
on HDFS. The path can be found out by looking at tez.lib.uris in
tez-site.xml. You could replace the file, or upload a new one and modify
tez-site to point to this.
At the moment, the client is using 0.8, while the AM is using 0.7 (which
comes from HDFS)
On Mon, Feb 22, 2016 at 4:14 AM, Dharmesh Kakadia <dh...@gmail.com>
wrote:
> It looks like those classes are added after 0.7 version of Tez. I got past
> that by using Tez version 0.8. May be we should add this to
> https://cwiki.apache.org/confluence/display/Hive/Hive-Tez+Compatibility to
> help others. or someone can give me permission to edit the wiki - my id is
> dharmesh.kakadia:)
>
> But now, I am stuck into "Incompatible versions found, clientVersion=0.8.2,
> AMVersion=0.7.0" error. I tried also
> setting tez.am.disable.client-version-check=true but still the same
> problem. Any pointer would help. Also, where does tez AM Jar is picked from
> ?
>
> Thanks,
> Dharmesh
>
> On Monday, February 22, 2016, Dharmesh Kakadia <dh...@gmail.com>
> wrote:
>
> > Hi,
> >
> > The hive 2.0 is exciting. Thanks for the great work !
> > I am getting the following error while giving it a try with tez 0.7.
> >
> > java.lang.NoClassDefFoundError:
> > org/apache/tez/serviceplugins/api/TaskScheduler
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.<clinit>(TezSessionState.java:93)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.createAndInitSession(TezSessionPoolManager.java:209)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getNewSessionState(TezSessionPoolManager.java:258)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:238)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:415)
> > at
> >
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:343)
> > at
> > org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:134)
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158)
> > at
> >
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101)
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840)
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584)
> > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
> > at
> > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
> > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:601)
> > at org.apache.hadoop.util.RunJar.run(RunJar.java:222)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.tez.serviceplugins.api.TaskScheduler
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> > ... 38 more
> > FAILED: Execution Error, return code -101 from
> > org.apache.hadoop.hive.ql.exec.tez.TezTask.
> > org/apache/tez/serviceplugins/api/TaskScheduler
> >
> > I understand that its not able to find a Tez class from the classpath,
> but
> > could not understand why. I am using the same tez binaries with hive 1.2
> > without any troubles. Is there are a required minimum version of Tez
> that
> > hive 2.0 supports? I downloaded 2.0 release from the website and using it
> > with 0.7 tez. Cluster is running latest hadoop/yarn.
> >
> > Thanks,
> > Dharmesh
> >
> >
>