You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Bitfox <bi...@bitfox.top> on 2022/03/28 04:21:07 UTC
Hive 3 with tez issue
When I had this config in hive-env.sh:
export
HADOOP_CLASSPATH=/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:$HADOOP_CLASSPATH
and when I started Hive I got the error:
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:304)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
at
org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
But when I removed that line in hive-env.sh and started hive again, it just
worked.
hadoop-3.3.2
hive-3.1.2
tez-0.10.1
java version "1.8.0_321"
All of them were installed in a local node for development purposes.
Please help with this issue. Thanks.
Bitfox
Re: Hive 3 with tez issue
Posted by Aaron Grubb <aa...@kaden.ai>.
Hive 3.1.2 is built with the 0.9.x branch of Tez. For information on which versions of which library you should use, you should check here: https://github.com/apache/hive/blob/rel/release-3.1.2/pom.xml
On Mon, 2022-03-28 at 17:08 +0800, Bitfox wrote:
Or, is there a standard installation guide for integration tez and hive3?
Thank you.
On Mon, Mar 28, 2022 at 12:21 PM Bitfox <bi...@bitfox.top> wrote:
When I had this config in hive-env.sh:
export HADOOP_CLASSPATH=/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:$HADOOP_CLASSPATH
and when I started Hive I got the error:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:304)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
But when I removed that line in hive-env.sh and started hive again, it just worked.
hadoop-3.3.2
hive-3.1.2
tez-0.10.1
java version "1.8.0_321"
All of them were installed in a local node for development purposes.
Please help with this issue. Thanks.
Bitfox
Re: Hive 3 with tez issue
Posted by Bitfox <bi...@bitfox.top>.
Or, is there a standard installation guide for integration tez and hive3?
Thank you.
On Mon, Mar 28, 2022 at 12:21 PM Bitfox <bi...@bitfox.top> wrote:
> When I had this config in hive-env.sh:
>
> export
> HADOOP_CLASSPATH=/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:$HADOOP_CLASSPATH
>
>
>
> and when I started Hive I got the error:
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:304)
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289)
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172)
>
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
>
> at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
>
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
>
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
>
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
>
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
>
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
>
> at
> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
>
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
>
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
>
>
>
>
>
> But when I removed that line in hive-env.sh and started hive again, it
> just worked.
>
>
> hadoop-3.3.2
>
> hive-3.1.2
>
> tez-0.10.1
>
> java version "1.8.0_321"
>
>
> All of them were installed in a local node for development purposes.
>
>
> Please help with this issue. Thanks.
>
> Bitfox
>
>
>
Re: Hive 3 with tez issue
Posted by Bitfox <bi...@bitfox.top>.
Or, is there a standard installation guide for integration tez and hive3?
Thank you.
On Mon, Mar 28, 2022 at 12:21 PM Bitfox <bi...@bitfox.top> wrote:
> When I had this config in hive-env.sh:
>
> export
> HADOOP_CLASSPATH=/opt/tez/conf:/opt/tez/*:/opt/tez/lib/*:$HADOOP_CLASSPATH
>
>
>
> and when I started Hive I got the error:
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:304)
>
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:289)
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172)
>
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
>
> at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
>
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
>
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
>
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
>
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
>
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
>
> at
> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
>
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
>
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
>
>
>
>
>
> But when I removed that line in hive-env.sh and started hive again, it
> just worked.
>
>
> hadoop-3.3.2
>
> hive-3.1.2
>
> tez-0.10.1
>
> java version "1.8.0_321"
>
>
> All of them were installed in a local node for development purposes.
>
>
> Please help with this issue. Thanks.
>
> Bitfox
>
>
>