You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Bala Krishna Gangisetty <ba...@altiscale.com> on 2014/07/18 02:44:51 UTC
HiveServer2 and Tez
Trying to use Tez through HiveSever2 client (beeline). But I was not
successful (pasted error message below). However Tez jobs can be run
successfully through Hive but not through HiveServer2 client. Is there any
specific configuration required for HiveServer2 in the context of Tez? Most
likely, the issue is because of not finding Tez jars in the HiveServer2
classpath. Appreciate any pointer to the HiverServer2 and Tez
configuration.
-- Bala G.
P.S:
Error log:
2014-07-17 23:39:05,353 WARN thrift.ThriftCLIService
(ThriftCLIService.java:ExecuteStatement(364)) - Error executing statement:
java.lang.RuntimeException: java.lang.NoClassDefFoundError:
org/apache/tez/dag/api/SessionNotRunning
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:84)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:500)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy15.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:234)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:358)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
RE: HiveServer2 and Tez
Posted by Bikas Saha <bi...@hortonworks.com>.
Do jps for HS2 and then use ps to figure out the classpath.
Bikas
*From:* Bala Krishna Gangisetty [mailto:bala@altiscale.com]
*Sent:* Thursday, July 17, 2014 6:57 PM
*To:* user@tez.incubator.apache.org
*Subject:* RE: HiveServer2 and Tez
Could you suggest how to verify and set HS2 classpath?
Sent from my Nexus
On Jul 17, 2014 6:37 PM, "Bikas Saha" <bi...@hortonworks.com> wrote:
Does your classpath for HS2 have Tez libraries in it? Check you Hive CLI
classpath and see if there is any difference.
Bikas
*From:* Bala Krishna Gangisetty [mailto:bala@altiscale.com]
*Sent:* Thursday, July 17, 2014 5:45 PM
*To:* user@tez.incubator.apache.org
*Subject:* HiveServer2 and Tez
Trying to use Tez through HiveSever2 client (beeline). But I was not
successful (pasted error message below). However Tez jobs can be run
successfully through Hive but not through HiveServer2 client. Is there any
specific configuration required for HiveServer2 in the context of Tez? Most
likely, the issue is because of not finding Tez jars in the HiveServer2
classpath. Appreciate any pointer to the HiverServer2 and Tez
configuration.
-- Bala G.
P.S:
Error log:
2014-07-17 23:39:05,353 WARN thrift.ThriftCLIService
(ThriftCLIService.java:ExecuteStatement(364)) - Error executing statement:
java.lang.RuntimeException: java.lang.NoClassDefFoundError:
org/apache/tez/dag/api/SessionNotRunning
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:84)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:500)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy15.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:234)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:358)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: HiveServer2 and Tez
Posted by Bala Krishna Gangisetty <ba...@altiscale.com>.
Could you suggest how to verify and set HS2 classpath?
Sent from my Nexus
On Jul 17, 2014 6:37 PM, "Bikas Saha" <bi...@hortonworks.com> wrote:
> Does your classpath for HS2 have Tez libraries in it? Check you Hive CLI
> classpath and see if there is any difference.
>
>
>
> Bikas
>
>
>
> *From:* Bala Krishna Gangisetty [mailto:bala@altiscale.com]
> *Sent:* Thursday, July 17, 2014 5:45 PM
> *To:* user@tez.incubator.apache.org
> *Subject:* HiveServer2 and Tez
>
>
>
> Trying to use Tez through HiveSever2 client (beeline). But I was not
> successful (pasted error message below). However Tez jobs can be run
> successfully through Hive but not through HiveServer2 client. Is there any
> specific configuration required for HiveServer2 in the context of Tez? Most
> likely, the issue is because of not finding Tez jars in the HiveServer2
> classpath. Appreciate any pointer to the HiverServer2 and Tez
> configuration.
>
>
>
> -- Bala G.
>
>
>
> P.S:
>
>
>
> Error log:
>
>
>
> 2014-07-17 23:39:05,353 WARN thrift.ThriftCLIService
> (ThriftCLIService.java:ExecuteStatement(364)) - Error executing statement:
>
> java.lang.RuntimeException: java.lang.NoClassDefFoundError:
> org/apache/tez/dag/api/SessionNotRunning
>
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:84)
>
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
>
> at
> org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:415)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>
> at
> org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:500)
>
> at
> org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
>
> at com.sun.proxy.$Proxy15.executeStatementAsync(Unknown Source)
>
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:234)
>
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:358)
>
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
>
> at
> org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
>
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
>
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:744)
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
RE: HiveServer2 and Tez
Posted by Bikas Saha <bi...@hortonworks.com>.
Does your classpath for HS2 have Tez libraries in it? Check you Hive CLI
classpath and see if there is any difference.
Bikas
*From:* Bala Krishna Gangisetty [mailto:bala@altiscale.com]
*Sent:* Thursday, July 17, 2014 5:45 PM
*To:* user@tez.incubator.apache.org
*Subject:* HiveServer2 and Tez
Trying to use Tez through HiveSever2 client (beeline). But I was not
successful (pasted error message below). However Tez jobs can be run
successfully through Hive but not through HiveServer2 client. Is there any
specific configuration required for HiveServer2 in the context of Tez? Most
likely, the issue is because of not finding Tez jars in the HiveServer2
classpath. Appreciate any pointer to the HiverServer2 and Tez
configuration.
-- Bala G.
P.S:
Error log:
2014-07-17 23:39:05,353 WARN thrift.ThriftCLIService
(ThriftCLIService.java:ExecuteStatement(364)) - Error executing statement:
java.lang.RuntimeException: java.lang.NoClassDefFoundError:
org/apache/tez/dag/api/SessionNotRunning
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:84)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:500)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy15.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:234)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:358)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.