You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Hilmi Egemen Ciritoğlu <hi...@gmail.com> on 2017/05/08 15:40:21 UTC

Running hive jobs on Tez Engine

Hi all,

So far I can run word count application on tez. But when I try to run hive
on tez, (of course after setting "hive.execution.engine=tez"), I got
following exception that I couldn't figure out even if I couldn't see any
post related to my issue on google.

Any help will be greatly appreciated.

Hadoop version: 2.7.2
Hive version: 1.2.2
Tez version: 0.8.4

Please let me know if you need any further information.

Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/tez/runtime/api/Event
    at
org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsDependentOptimizations(TezCompiler.java:270)
    at
org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:124)
    at
org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:102)
    at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10210)
    at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:210)
    at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1145)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1193)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
    at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
    at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException:
org.apache.tez.runtime.api.Event
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 24 more

Re: Running hive jobs on Tez Engine

Posted by Hilmi Egemen Ciritoğlu <hi...@gmail.com>.
Hi Harish and Kuhu,

Thanks for quick and great response !

I already set tez jars classpath before. But some how, it's deleted !
After setting again everything is cool...

Cheers...

Regards,
Egemen

2017-05-08 16:50 GMT+01:00 Kuhu Shukla <ks...@yahoo-inc.com>:

> Hi Hilmi,
>
> Tez jars are required in the classpath, typically configured though
> tez.lib.uris config property.
>
> Thanks and Regards,
> Kuhu
>
>
> On Monday, May 8, 2017 10:48 AM, Harish JP <hj...@hortonworks.com> wrote:
>
>
> You are missing tez jars in the classpath. Most likely you are missing
> tez-site.xml if this is a custom setup. Please follow instructions to setup
> client and hadoop environment: https://tez.apache.org/install.html.
>
> On 08-May-2017, at 9:10 PM, Hilmi Egemen Ciritoğlu <
> hilmi.egemen.ciritoglu@gmail.com> wrote:
>
> Hi all,
>
> So far I can run word count application on tez. But when I try to run hive
> on tez, (of course after setting "hive.execution.engine=tez"), I got
> following exception that I couldn't figure out even if I couldn't see any
> post related to my issue on google.
>
> Any help will be greatly appreciated.
>
> Hadoop version: 2.7.2
> Hive version: 1.2.2
> Tez version: 0.8.4
>
> Please let me know if you need any further information.
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/tez/runtime/api/Event
>     at org.apache.hadoop.hive.ql.parse.TezCompiler.
> runStatsDependentOptimizations(TezCompiler.java:270)
>     at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(
> TezCompiler.java:124)
>     at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(
> TaskCompiler.java:102)
>     at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:10210)
>     at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:210)
>     at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.
> analyze(BaseSemanticAnalyzer.java:227)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
>     at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1145)
>     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1193)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
> CliDriver.java:213)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(
> CliDriver.java:376)
>     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(
> CliDriver.java:736)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.ClassNotFoundException: org.apache.tez.runtime.api.
> Event
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>     ... 24 more
>
>
>
>
>

Re: Running hive jobs on Tez Engine

Posted by Kuhu Shukla <ks...@yahoo-inc.com>.
Hi Hilmi,
Tez jars are required in the classpath, typically configured though tez.lib.uris config property. 
Thanks and Regards,Kuhu 

    On Monday, May 8, 2017 10:48 AM, Harish JP <hj...@hortonworks.com> wrote:
 

 You are missing tez jars in the classpath. Most likely you are missing tez-site.xml if this is a custom setup. Please follow instructions to setup client and hadoop environment: https://tez.apache.org/install.html.
On 08-May-2017, at 9:10 PM, Hilmi Egemen Ciritoğlu <hi...@gmail.com> wrote:
Hi all,
So far I can run word count application on tez. But when I try to run hive on tez, (of course after setting "hive.execution.engine=tez"), I got following exception that I couldn't figure out even if I couldn't see any post related to my issue on google.
Any help will be greatly appreciated. 
Hadoop version: 2.7.2Hive version: 1.2.2Tez version: 0.8.4
Please let me know if you need any further information.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tez/runtime/api/Event
    at org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsDependentOptimizations(TezCompiler.java:270)
    at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:124)
    at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:102)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10210)
    at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:210)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1145)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1193)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.tez.runtime.api.Event
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 24 more

Re: Running hive jobs on Tez Engine

Posted by Harish JP <hj...@hortonworks.com>.
You are missing tez jars in the classpath. Most likely you are missing tez-site.xml if this is a custom setup. Please follow instructions to setup client and hadoop environment: https://tez.apache.org/install.html.

On 08-May-2017, at 9:10 PM, Hilmi Egemen Ciritoğlu <hi...@gmail.com>> wrote:

Hi all,

So far I can run word count application on tez. But when I try to run hive on tez, (of course after setting "hive.execution.engine=tez"), I got following exception that I couldn't figure out even if I couldn't see any post related to my issue on google.

Any help will be greatly appreciated.

Hadoop version: 2.7.2
Hive version: 1.2.2
Tez version: 0.8.4

Please let me know if you need any further information.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tez/runtime/api/Event
    at org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsDependentOptimizations(TezCompiler.java:270)
    at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:124)
    at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:102)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10210)
    at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:210)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1145)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1193)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.tez.runtime.api.Event
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 24 more